[TYPO3] crawler extension & external non-typo3 sites

[Dmitry Dulepov [typo3]]

Hi!

Elijah wrote:
> It seems that the typo3 crawler could only go through similar cached 
> typo3 sites (pls correct me if I'm wrong) but what about other dynamic 
> non-typo3 sites?
> 
> I've just tested it on one external non-typo3 link and made it crawl 
> through 3 levels and it only fetched the top level link and never went 
> through the links. But while testing it on a cached typo3 site it 
> traversed through the page tree nicely.

I went to typo3.org and read the very beginning of the manual for crawler. It contains the answer to your question. Did you bother RTFM before asking? :)

-- 
Dmitry Dulepov
TYPO3 core team
Web: http://typo3bloke.net/
Skype: callto:liels_bugs
"Nothing is impossible. There are only limits to our knowledge"