[TYPO3-dev] indexed_search kills server

[Ernesto Baschny [cron IT]]

Oliver Rowlands wrote: on 21.05.2008 11:45:

> The 21,263,272 Rows_examined is where your problems are coming from.
> 
> The bottom line is 'indexed_search' does not scale. It's great for a 100 
> page site but beyond that you will be better off using an external crawler.
> 
> You might want to give Dimitry's 'mnogosearch' extension a try, it is 
> far more scalable and efficient than 'indexed_search' will ever be.
> 
> If you are stuck with 'indexed_search' you could try enabling 
> '$join_pages' in the tx_indexedsearch class if you are not bothered by 
> 'extendToSubpages':
> 
> http://typo3.org/documentation/document-library/extension-manuals/doc_indexed_search/current/view/1/6/#id3520912 

btw this can be configured from TypoScript since 4.2:

search.skipExtendToSubpagesChecking
  If set to false (default), on each search the complete page tree will
  be transversed to check which pages are accessible, so that the
  extendToSubpages can be considered. This will work with a limited
  number of page-ids (which means most sites), but will result in slow
  performance on huge page trees.

  If set to true, then the final result rows are joined with the pages
  table to select pages that are currently accessible. This will speed up
  searching in very huge page trees, but on the other hand
  extendToSubpages will NOT be taken into account!

Default: 0

That won't help on that particular query, as it is a single query for 
word occurrences in a huge data-set (searching with "like"). I guess 
either the content amount is really huge, in which case you are probably 
right about indexed_search scalability (will be better in 4.3 with 
FULLTEXT support), but I guess that the problem has another cause: The 
same content being cached multiple times and thus being indexed multiple 
times: e.g. same page, multiple cHash'es because some plugin generating 
"endless" links to the same page (e.g. might happen with the "cal"). Try 
to find out which page has multiple indexed version (use Web>Info module 
for that) and figure out what might cause it.

Cheers,
Ernesto