[TYPO3-core] Supporting Thai in the index searach

[Martin Kutschker]

Hi!

To support Thai letters it is necessary to add the character ranges for letters (alpha, not CJK!) and digits into class.lexer.php:

($cp >= 0xE01 && $cp <= 0xE4E && $cp != 0xE3F) // letters
($cp >= 0xE50 && $cp <= 0xE59) // digits

The letters exclude the currency symbol baht.

The numerals may be omitted as the the sample texts I have use Roman digits.

May I add this to TYPO3_4-0? It's not really a bug fix, OTOH it's not much of a change/feature.

Perhaps we should add support for all scripts of TYPO3 BE languages (eg Burmese/Myanmar).

Masi

PS: Hiragana and Katakana should not be reported as "CJK" signs, but as "alpha" letters. They represent syllables, not ideas, therefore you can *spell* words with them.