[TYPO3-english] TYPO3 and character encoding problems

[Pero Peric]

On 7.7.2014. 21:06, Jigal van Hemert wrote:
> Hi,
>
> On 7-7-2014 16:26, Pero Peric wrote:
>> I tried many things but nothing works. In this conversion scripts it is
>> always assumed that tables/columns are in Latin1 but all my table
>> definitions are UTF8.
>
> When 4.7 was published I wrote a script to convert the database, table
> and columns to UTF-8. It also tries to detect the most common problem of
> double UTF-8 encoded data and attempts to solve this by making MySQL
> re-interpret the data.
> It won't work with 6.x (but I haven't seen 6.x installations with
> databases that aren't fully UTF-8).
>
> 1. Backup your database (really!! make a backup)
> 2. See: step 1.
> 3. Empty tables you don't really need. cache_* tables, sys_log, can be
> very large and if you can live without the data the conversion is a lot
> faster.
> 4. Download http://www.typo3coder.nl/fileadmin/downloads/fixdb.zip
> 5. Put the script inside that archive in /typo3/install/
> 6. Run /typo3/install/fixdb.php in your browser (if there is no
> ENABLE_INSTALL_TOOL file you'll see a warning)
>
> 7. Check if the problem is solved. If no, restore database backup; in
> any case remove fixdb.php !!
>
> Hope it helps.

I tried it and i get:

Database character set is UTF-8
Character set of all tables is UTF-8
Character set of all columns is UTF-8
All columns seem to contain correctly encoded data

Like all is ok, but it's not. I can see that hexadecimal char values are 
not right in DB.

What I would like to know - what does forceCharset setting do? If i set 
this to blank in 4.4.0 i get garbage characters. If i set it to UTF-8 
all is fine. I searched to find how TYPO3 convert this but didn't find 
anything :-(

Regards.