[TYPO3] crawler

Rico Moorman rico.moorman at gmail.com
Thu Jan 19 21:14:29 CET 2006


Well actually I was wondering about a way to make it work just out of 
the backend or/and to make it work with a script thats called via a cron 
job...
Maybe I got it wrong but thats some functionality the "crawler" is going 
to offer

actually it just has to be a script ...getting all pages (page ids) that 
are visible in the FE ...
and then one by one "crawling" along the site (index.php?id=1 then 
index.php?id=31) using maybe wget requests or something like that so 
that the index could be refreshed on a regular basis or by clicking some 
button in the BE


Dmitry Dulepov schreef:
> Hi!
> 
> Rico Moorman wrote:
> 
>>dear list,
>>
>>Searching for some help about refreshing the index of the indexed search
>>engine on a regular basis I would like to drop a question here.
>>I am trying to make the crawler extension work ... is there some way to
>>do that and to have it indexing the page regularly?
>>
>>If it still needs some programming to make it work actually I would also
>>be willing to help developing the extension further.
> 
> 
> I use Xenu's Link Sleuth. While it checks site for dead links, typo3
> builds its index :)
> 
> Dmitry.



More information about the TYPO3-english mailing list