[TYPO3-english] Crawler and external documents

[Dmitry Dulepov]

Hi!

Claudio Strizzolo wrote:
> I'm trying to set up the crawler extension in order to index all the pages
> in the site and the external documents (/fileadmin/...) linked by anchors 
> in the pages.
> I read some documentation, included http://wiki.typo3.org/index.php/
> Ext_crawler and almost everything works: the pages are correctly indexed, 
> and the external documents are recognized. In the Crawler Log they are 
> listed in separate rows under the page which points to them.
> However, their status is ".." and their contents are not indexed. If I 
> click on the "Read" icon (it looks more like a reload icon, imho) the 
> content is correctly indexed and the status becomes "OK", but I could not 
> find a way to get this automatically through the crawler.

We lost lots of time while trying to index external documents with crawler/indexed search. In the end we just switched to mnogosearch, which does it much better. With crawler/indexed search you never know if something will be indexed or not...

-- 
Dmitry Dulepov
TYPO3 core team
"Sometimes they go bad. No one knows why" (Cameron, TSCC, "Dungeons&Dragons")