[TYPO3-Solr] Fatal error in solr with some search words

[Ingo Renner]

Am 29.08.12 10:36, schrieb Jigal van Hemert:

Hi Jigal,

> Using the query from Solr (thanks devlog :-) ) I got the error:

;)

> HTTP Status 500 -
> org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token
> <searchword> exceeds length of provided text sized 25

we also stumbled over this one already with one or two customers but 
could never find out what is causing that erroe.

> I found a few bug reports about this, the main one is
> https://issues.apache.org/jira/browse/LUCENE-2208
>
> The problem seems to be with the HTMLStripCharFilter which is used for
> highlighting.

Actually I couldn't find any usage of that class in the extension's 
configuration files. Do you have any exact pointers I might have overlooked?

Currently from quickly looking over the mentioned issue I can only 
imagine it's used internally in the highlighter.

> This is apparently solved with
> https://issues.apache.org/jira/browse/LUCENE-3690
>
> Is it possible to somehow use this fix with solr <schema
> name="tx_solr-2-5-0--20120214" version="1.4"> ? Is there an easy upgrade
> possible to a newer version which is still compatible with this EXT:solr
> and the indexes?

As the issue mentions it is solved in Solr 3.6, so you'd need to upgrade 
to that version. We don't currently explicitly support 3.6 yet, but it 
should not be an issue. (We rather would like to directly move to 4.0 
when it ships.)

solrconfig.xml specifies the luceneMatchVersion so that a Solr 3.6 
server would behave like a 3.5 version - pretty much like our 
compatVersion in TYPO3.

Thanks for digging into this!


HTH
Ingo

-- 
Ingo Renner
TYPO3 Core Developer, Release Manager TYPO3 4.2, Admin Google Summer of Code

TYPO3 - Open Source Enterprise Content Management System
http://typo3.org

Apache Solr for TYPO3 -
Open Source Enterprise Search meets Open Source Enterprise CMS
http://www.typo3-solr.com