[TYPO3] crawler questions

[Steffen Kamper]

Hi Lars,

glad to see i'm not alone with this :-)

> I had to fight with the re-indexing while developing. All jobs ran fine, 
> but I forgot that the re-indexing was set for the next hour, so the 
> crawler did its all 5 minute job but ignored new records. If you reindex 
> all 24hours you could wait a long time. Just make sure reindexing is 
> left blank each time you test.
>
ok, i will try

> Database Fields which are indexed were also something I forgot at first. 
> In the FAQ extension they are q and a - not title and text.
> 
> Last is how many records are indexed. May it is just not finished ?
>
yes, indeed it's unclear, i have to look at source code

>> So some questions for the first:
>>
>> 1) Where should the record for crawling pages should be placed? Kasper 
>> put it in a storage folder, i also tried on root page
> 
> I thought they had to be stored on the pages where the output records 
> are rendered.
this is true for eg Database records, but not for pagetree. Seems that 
it makes no difference for this record.

> 
>> 2) What's about the maximum level "3" configured in record, as my page 
>> tree has more levels?
> 
> You are so right :-(
> Nice to have more levels in a furture indexed_search release.
>
i will hack the TCA of crawler ext to add more levels

> 
>> 3) record has time setting for queuing the record, but it crawls the 
>> pages in each run, does it have no influence?
> 
> As far as I see in my installations it only reindexes if something new 
> is on the page or normal re-caching is needed. As for the load, I dont 
> know because there aren't that many pages on my installations.
> 
i puzzle about this reindexing. Messages in crawler log saying something 
about reindexing, but i didn't understand the meaning here, so i have to 
try&error

>> 4) i tried to configure crawler to use mm_forum as data table. now the 
>> table with the posts is kind of mm-table so i can't build the Getvars 
>> without including the thread table, is there any workaround for such 
>> situation?
> 
> sorry ... no advice here :-(
> 
np - i will investigate further and post here if i found something 
interesting about.

vg Steffen