[TYPO3] crawler questions

[Lars Haendler]

Hi Steffen,

Steffen Kamper wrote:
> Hi,
> 
> i try to configure the crawler proper on a server and have some problems.
> 
> I read documentations of indexed_search and crawler, looked to the 
> podcast of Kasper and used the wiki page for configuration examples.
> 
> In my local installation all works like in Kasper's video, but on the 
> life server it doesn't.
> Looking to the crawler log all seems to work fine, but it seems that the 
> urls are not crawled proper as search results are not show all existing 
> results. Does anyone has good expieriences with crawler?

I had to fight with the re-indexing while developing. All jobs ran fine, 
but I forgot that the re-indexing was set for the next hour, so the 
crawler did its all 5 minute job but ignored new records. If you reindex 
all 24hours you could wait a long time. Just make sure reindexing is 
left blank each time you test.

Database Fields which are indexed were also something I forgot at first. 
In the FAQ extension they are q and a - not title and text.

Last is how many records are indexed. May it is just not finished ?

> So some questions for the first:
> 
> 1) Where should the record for crawling pages should be placed? Kasper 
> put it in a storage folder, i also tried on root page

I thought they had to be stored on the pages where the output records 
are rendered.

> 2) What's about the maximum level "3" configured in record, as my page 
> tree has more levels?

You are so right :-(
Nice to have more levels in a furture indexed_search release.


> 3) record has time setting for queuing the record, but it crawls the 
> pages in each run, does it have no influence?

As far as I see in my installations it only reindexes if something new 
is on the page or normal re-caching is needed. As for the load, I dont 
know because there aren't that many pages on my installations.

> 4) i tried to configure crawler to use mm_forum as data table. now the 
> table with the posts is kind of mm-table so i can't build the Getvars 
> without including the thread table, is there any workaround for such 
> situation?

sorry ... no advice here :-(

> vg Steffen

Regards
Lars