[Typo3] Chars in record fields are truncated in backend forms

[redacted user]

Note: This message is encoded in UTF-8, in case you've problems viewing
certain characters...

Hello there!

I'm currently experience some major problems with Typo3 and UTF-8.
A few words on my setup:

Typo3 v3.8.0
MySQL 4.1.13
PHP 4.3.11

Typo3 is configured with $TYPO3_CONF_VARS['BE']['forceCharset'] =
"utf-8" (backend and frontend puts out UTF-8 content),
$TYPO3_CONF_VARS['SYS']['multiplyDBfieldSize'] is unchanged (1). MySQL
runs completly in UTF-8 mode (client and server connection, all tables,
columns and the typo3 db itself is utf-8 with utf8-general-ci collation.
PHP has the default charset "utf-8" set in php.ini.

So whats the problem? I've configured some tables by extending the $TCA
hash in a self-written plugin. User input for these records works for
"normal" input, but if special characters, in our case german umlauts,
are put into the fields, these seem to be interpreted as two-byte digits
since the total length of the input field (which we check) disregards
chars after these "special chars". An example:

Consider the following $TCA snippet:

'plz' => Array (
 'exclude' => 0,
 'label' => 'LLL:EXT:<our_ext>/locallang_db.php:<our_ext>.plz',
 'config' => Array (
  'type' => 'input',
  'size' => '10',
  'max' => '10',
  'eval' => 'trim,required',
 )
),

The field "plz" should contain a maximum of 10 characters. This works
without problems for normal (ASCII) characters like a-zA-z0-9 aso.
But if e.g. only special chars are put in, like all 'ä', instead of
saving all 10 'ä' only 5 are saved and displayed after the save button
is clicked. Also, in DB only "äääää" is put in instead of "ääääääääää".
This is a blocker because the JS field check correctly counts 10
characters and not 20 like the backend seems to count.

Furthermore, another problem in the backend: Sharing utf8-encoded
language files in backend and frontend (this file is locallang_db.php).
While the display on the frontend works without a problem (the lang file
is loaded  there with $this->pi_loadLL() from the ..._pibase class), the
same language file screws up in the backend showing broken, double
byte'd characters, e.g. the little "ä" is displayed as "ä". Weird
enough database contents encoded in utf-8 are displayed CORRECTLY in the
backend.

I've read about using xml files instead of .php files which should be
always recognized properly as utf-8. While I could manage to load a test
xml file (utf8-encoded locallang_test.xml which "acts" as
locallang_db.php) for the frontend via
t3lib_div::readLLfile($pathToXmlFile, $this->LLkey), the characters
where printed out as ISO-8859-1 to the browser (at least when I switched
my browser's encoding from utf-8 to iso88591 the previous "?" chars were
converted to the proper characters). Furthermore I couldn't get the XML
file to work at all in the backend, no matter how hard I tried it.
Having a locallang_db.php and a locallang_db.xml file in the extension
dir apparently makes Typo3 to ignore the php file instantly. Since all
labels in the $TCA array still point to the php file I changed some
references to the xml file like

'label' => 'LLL:EXT:<our_ext>/locallang_db.xml:<our_ext>.plz',

but that didn't helped, still no labels were displayed. Same when I
named the xml file to locallang_temp.xml.


Any help is greatly appreciated!!!
Thanks,
Thomas.

-- 
itCampus Software- und Systemhaus GmbH
Leipzig - Halle - Wittenberg

Nonnenstraße 42
04229 Leipzig

t.keller at itcampus.de
Tel: +49. 341. 4 92 87 32
Fax: +49. 341. 4 92 87 01
GPG-Key: 0xB536A35F

http://www.itcampus.de

Amtsgericht Leipzig HRB 15872
Geschäftsführer Andreas Lassmann