[TYPO3] crawler_cli.phpsh not working?

[Ries van Twisk]

Hey,

I had problems to get it running at all.
First of all on this server for some reason I could't 'execute' the 
crawler in cli mode at all,
so I wrote a little helper script. SOmething in the line as
php -q path/to/the/crawler.cli.php

Second of all in cli mode the crawler could not figure out it's own path 
name
so I had to hard code it. The config file wasn't found.

After that was solved I had to go in the backend and prepare 'something' 
for crawler
mode. I wasn't really clear about that but following the crawler manual 
did the trick.

It some point when I started the the crawler it took about 2-3 minutes 
before my site was indexed.
When I started it right after wards it was fast and returned directly. I 
think it was not scheduled yet.
However I couldn't figure out clearly when it was scheduled for the next 
time. I need to test this I think.
The next schedule time stayed empty and I didn't understand why.
These where my first baby crawler steps........ I need to play around 
with it more.

I also need to badly tune indexed-search. For example if I look for the 
word 'downloads' on my site now
It will give me all pages back with the word download on the page. means 
all pages since he
get's it from the menu!!! When I go to typo3.org and do the same I 
basicly get the same results,
means a lot of the same pages so I am not sure if I can solve it... Or 
typo3 is misconfiguration... I don't know...
I need to play much more with that search engine I think, or I will 
install mnogosearch for this client.


So far.......

Ries



> Still no answer...
> Doesn't anybody know the answer or does nobody understand
> the problem?
>
> When running the the crawler_cli.phpsh no error messages are
> shown and nothing happens (crawler queue doesn't get processed).
>
> When adding some debug echo statements to the script I see
> that it doesn't seem to get past the 'init.php' line:
>
> --- crawler_cli.phpsh ---
> [...]
> echo "Debug 1\n";
> // Include init file:
> require(dirname(PATH_thisScript).'/'.$BACK_PATH.'init.php');
> echo "Debug 2\n";
> [...]
>
> This will result in "Debug 1" getting printed in the console
> but "Debug 2" doesn't. What could be wrong?
>
> Thanks
>
>   
>> I can't get the crawler to run from the command line.
>> Everything else seems to work fine. Entries are added to the
>> crawlers queue and when I press the "Run now" button in the
>> backend they get processed. However when I run the crawler
>> from the command line nothing happens. The "Last seen" time
>> does not get updated.
>>
>> I'm calling the script with the full path as shown in the
>> backend:
>> '/data/www/intraweb/typo3conf/ext/crawler/cli/crawler_cli.phpsh'.
>> No error messages are shown. I did create a backend user
>> named '_cli_crawler'. Does that user need any special
>> options set? What about the password for that user?
>>
>> I tried setting the PATH_thisScript variable manually to the
>> correct value, but that didn't help either.
>>
>>
>>     
> define('PATH_thisScript','/data/www/intraweb/typo3conf/ext/crawler/cli/crawler_cli.phpsh');
>   
>> Any advice?
>>
>> Thanks.
>>     
>
>
> jwittema at yahoo.com
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around 
> http://mail.yahoo.com 
> _______________________________________________
> TYPO3-english mailing list
> TYPO3-english at lists.netfielders.de
> http://lists.netfielders.de/cgi-bin/mailman/listinfo/typo3-english
>   


-- 
Ries van Twisk
Freelance Typo3 Developer
=== Private:
email: ries at vantwisk.nl
web:   http://www.rvantwisk.nl/freelance-typo3.html
skype: callto://r.vantwisk
=== Work:
email: ries at livetravelguides.com
web:   http://www.livetravelguides.com