[TYPO3] [Typo3] Indexed Search (CVS Version) + MySQL 4.1 + UTF-8 => no results with special chars

[Nikolas Hagelstein]

Hi, 
> My servers are running with this configuration:
> - BE/forceCharset in install tool is set to UTF-8
> - SYS/multiplyDBfieldSize is set to 2
> - all templates are UTF-8
> - all content is UTF-8 (because BE is UTF-8)
> - mysql runs with latin1
Dimitry, so what do you think about 
setting DB fields encoding to utf8  and
force DB collation(connection) to utf8 
(by adding a "set names utf8" to tslib).

Having db and collation in latin1 could 
get you into trouble when doing sql dump
import/export without an appropriate --default-charset during
export.

To bring a little light to charset "magic"
this is how things are imho working:

 latin1          
---------        latin1          utf8   
|dbfield| ----db connection--->|OUTPUT|
---------


 latin1          
---------        latin1          utf8   
|dbfield| <----db connection---|input|
---------


As you see everthing from and to DB is stored/fetched
"as it is". Charset interpretation is left to 
the last client instance (typo3back end or browser frontend).
The main disadvantage of this implementation is, that 
mysql does not "knows" what its content is of.
So native DB views/functions  are leading to unreadable
results.

Trouble occurs if the db connection is set to something
else than the enconding of the DB field. This will result
in charset conversion (done by mysql) which will lead to 
broken chars.

Imho it would be a good idea to improve the typo3 DB layer
to have nativ utf8 support on demand.

cheers,

          Nikolas