[TYPO3-core] RFC #7942: Enable UTF-8 by default

[Jigal van Hemert]

Hi,

On 10-11-2010 16:26, Martin Kutschker wrote:
> Am 10.11.2010 15:04, schrieb Jigal van Hemert:
>>
>>> So I would recommend a check of the default charset of the
>>> database and a check of all table columns for any existing tables
>>> within the DB.
>>
>> Not necessary at all.
>
> Care to elaborate?

To make the entire backend use UTF-8 there is no real need to have the
database, tables or columns use a utf8 charset or compatible collation.

If you try to use data which contains characters which cannot be
represented in the character set of the column (or table or database)
data loss may occur, but other than that everything is converted
automatically by MySQL.

So, if someone has a column with latin1 charset then it can easily store
Western European text coming from a UTF-8 TYPO3 backend. Comparing and
sorting will be done according to the collation of that column, unless
stated otherwise in the query. TYPO3 does not use any collation in its
queries, so the latter situation will not occur.

>
> My fear is that someone tries to install TYPO3 on a DB that contains
> (empty) TYPO3 tables (eg by using a schema dump to create it by
> hand). In this case you could end up with columns that are not utf8.

This is not a real problem. Setting "SET NAMES utf8" will ensure that 
the client (the mysql module in PHP) will communicate with MySQL using 
UTF-8 and that MySQL expects incoming data as UTF-8 and that MySQL will 
convert outgoing data to UTF-8.

So data from big5 tables will end up in UTF-8 encoding in PHP.

There may be cases where people specifically use a charset or collation. 
Suppose you have an Danish only website and want to ensure that sorting 
and comparing of titles is done according to Danish rules you could set 
the collation of those columns to utf8_danish_ci (or a similar collation 
compatible with the charset of the column).

-- 
Kind regards / met vriendelijke groet,

Jigal van Hemert
skype:jigal.van.hemert
msn: jigal at xs4all.nl
http://twitter.com/jigalvh