[Typo3-dev] extension "crawler"

[Kraft Bernhard]

Stefan Beylen wrote:

> i dont know much about crawlers nor cronjobs, but if i call every page 
> in lynx it should be indexed by the indexed search, right?


wget -m http://www.yoursite.com/

will make you life easier than fetching every single page. It creates a directory called
www.yoursite.com in the directory where it is called ... so you most probably want to
put it into a little shell script which changes directory to /tmp first (don't know in which directory
jobs executed from cron run)


greets,
Bernhard
-- 
Kraft Bernhard
MOKKA Medienagentur <http://www.mokka.at>
T: +43 - 1 - 895 33 33 - 50