[TYPO3-core] indexed_search patches

[Michael Stucki]

Hello,

during the last months I have made many changes to indexed search. This
results in quite a big patch. I have tested the critical parts for a long
time now and think that everything should be fine. However it's in any case
better if anybody reviews my changes.

If there's something we need to discuss (e.g. changed default values), I
suggest we do it right here.


So here is the list of changes:

- Fixed the page counter which was not working correct under some
circumstances. Needs review and probably some more testing...
See http://bugs.typo3.org/view.php?id67
Path was initiated by Chi Hoang


- Template support for indexed_search.
This works great, I already moved two of our customers to this new version.
See http://bugs.typo3.org/view.php?id†1
Big thanks to Udo von Eynern


- Fixed a problem with continuing indexing of external documents. This just
did not work - if a file is fetched from the cache, it's content was not
indexed anymore. Fixed this by moving the indexer hook out of the page
generation function, right after it is stored/fetched from the cache.


- Completely removed the Ruby/OOoExtract functionality. Use "unzip" instead.
In my opinion, this is much simpler than using a tool that is not part of
any well-known Linux distribution <=

- Replaced is_file calls with a new function: Test if safe_mode or
open_basedir are set and return true if a custom test has worked.


- The behavior of the result browser has been changed. It works more
Google-like now.


- If an external file is too large (e.g. TSref), then the lexer needs too
long for processing the words. To work around this, I have hardcoded that
indexing of documents with $contentArr['body'] longer than 100000
characters will be skipped. Of course the better solution is to make the
lexer class (much!) faster...


Many smaller changes:
- The default number of results per page will be changed from 20 to 10
- Fix the hit counter. See http://bugs.typo3.org/view.php?id67
- Cut HTTP references after the question mark (too long URLs can break a
  design otherwise!)
- Generally: preg_replace is faster than ereg_replace! - thanks to the
  Austrian UG (can't remember who it was) for reporting this.
- New TS properties: See section "02_setup_txt.diff" below
- New TLO "config" properties "index_keywords" and "index_description". Both
  will be disabled by default (which is not the same like the current
  behavior, though I think that nobody really wants this).
- Renamed procesWordsInArrays() to processWordsInArrays()


==
Attached are the patched files, split up file by file which makes it a
little easier to follow:

01, 03 and 04 are very minor things.

02 adds some new TS properties. See comments in 05_plugin.diff for details.
   show.forbiddenRecords    show.alwaysShowPageLinks    show.advancedSearchLink    show.mediaList    show.resultNumber    _DEFAULT_PI_VARS.type    _DEFAULT_PI_VARS.flat 
   Do you agree with these changes?

05 is the FE plugin. For details, watch the comments in the diff file.

06 are the indexer classes.

10 contains changes outside of indexed_search, needed for fixing the
continuing stuff...

*tmpl are two new templates, written by Udo. The traditional template looks
exactly like the current one and should be used by default.


Regards, michael
-- 
Use a newsreader! Check out
http://typo3.org/community/mailing-lists/use-a-news-reader/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 01_considerations.diff
Type: text/x-diff
Size: 1331 bytes
Desc: not available
Url : http://lists.netfielders.de/pipermail/typo3-team-core/attachments/20050928/d5e9a806/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 02_setup_txt.diff
Type: text/x-diff
Size: 1342 bytes
Desc: not available
Url : http://lists.netfielders.de/pipermail/typo3-team-core/attachments/20050928/d5e9a806/attachment-0001.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 03_modfunc.diff
Type: text/x-diff
Size: 698 bytes
Desc: not available
Url : http://lists.netfielders.de/pipermail/typo3-team-core/attachments/20050928/d5e9a806/attachment-0002.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 04_locallang.diff
Type: text/x-diff
Size: 785 bytes
Desc: not available
Url : http://lists.netfielders.de/pipermail/typo3-team-core/attachments/20050928/d5e9a806/attachment-0003.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 05_plugin.diff
Type: text/x-diff
Size: 30541 bytes
Desc: not available
Url : http://lists.netfielders.de/pipermail/typo3-team-core/attachments/20050928/d5e9a806/attachment-0004.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 06_indexer.diff
Type: text/x-diff
Size: 28402 bytes
Desc: not available
Url : http://lists.netfielders.de/pipermail/typo3-team-core/attachments/20050928/d5e9a806/attachment-0005.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 10_index_multiple.diff
Type: text/x-diff
Size: 4018 bytes
Desc: not available
Url : http://lists.netfielders.de/pipermail/typo3-team-core/attachments/20050928/d5e9a806/attachment-0006.bin 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.netfielders.de/pipermail/typo3-team-core/attachments/20050928/d5e9a806/attachment.html 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.netfielders.de/pipermail/typo3-team-core/attachments/20050928/d5e9a806/attachment.htm