[TYPO3-dev] Weird Upgrade Problem 4.1.x -> 4.2.x

[Martin Kutschker]

Peter Russ schrieb:
> --- Original Nachricht ---
> Absender:   Martin Kutschker
> Datum:       18.03.2009 08:32:
>> Peter Russ schrieb:
>>> --- Original Nachricht ---
>>> Absender:   Martin Kutschker
>>> Datum:       18.03.2009 08:16:
>>>> Peter Russ schrieb:
>>>>> Thanks Ben, but we don't have problem to convert as customer would
>>>>> like
>>>>> to stay with iso-8859-1 (latin1). The point is that we loose
>>>>> information
>>>>> when upgrading ...
>>>> You say that the Database runs in utf8 but TYPO3 uses iso-8859-1?
>>>>
>>>> What does that mean in terms of setup (Mysql and TYPO3)?
>>>>
>>>> Anyway, what I wrote about BLOB to TEXT conversion could be also a
>>>> problem in this case. If Mysql really stores the data in utf8 (but
>>>> talks
>>>> to the client in iso-8859-1) then the data BLOB will not be
>>>> converted to
>>>> utf-8 when the data type is changed. OTOH the on-the-fly conversion
>>>> from
>>>> utf8 (storage format) to iso-8859-1 (client format) will fail (as
>>>> iso-8859-1 is nt valid in utf-8).
>>>>
>>>> Masi
>>> Masi I would agree, if that would be also a problem with version 4.1.10
>>> of TYPO3. BUT we get the problem after update, i.e. after the
>>> ALTER table .... TEXT
>>> converting blob to something different that blob performed by
>>> install->compare
>>
>> Would you please post your TYPO3 (forceCharset, dbInit) and Mysql (SHOW
>> VARIABLES LIKE 'character_set%';) settings?
>>
>> I'm not sure if I understand you, but I have the feeling that you don't
>> understand what the difference between TEXT and BLOB is.
>>
>> Masi
> 
> Perhaps this might help you to understand:

Mysql stores data in utf8 (because of "character_set_database") but
accepts and sends data in latin1 ("character_set_client",
"character_set_results", "character_set_connection").

All columns know to contain text data (which have a charset) are
therefore automatically converted for each query. CHAR, VARCHAR and TEXT
are such column types.

Other fields including binary types (like BLOB) are not converted.

What happened to you is that TYPO3 4.1 wrote latin1 into the BLOB. Which
was fine as no conversion took place.

Now in TYPO3 4.2 the type was changed to TEXT. So Mysql thinks that your
old data (which are *stored* as binary data but actually in latin1) is
in utf8. Now it tries to convert (see above) to convert real latin1 from
utf8 to latin1 which must fail.

So it will work again if you create a new template, because then the
charset is correctly sent and converted.

But I suggest what you did already: use the same charset both for
storage and connection.

Alternatively you could have simply skipped to BLOB to TEXT conversion
in the installer.

Masi