[TYPO3-v4] [XLIFF] Support plurals

[Xavier Perseguers]

Hi,

With TYPO3 4.6 we refactored language handling and switched to XLIFF for
localization files. The great benefit of XLIFF is to be _normalized_ and
the native format of Pootle, the software that runs our translation
server on [1]. This ensures that 3rd party software may deal with our
localization files instead of having to rely on some own transformation
back and forth when dealing with them (export/import to Excel files for
instance).

As you may have noticed, for the time being, an ll-XML or an XLIFF file
basically are similar and in fact, it is just a matter of some XSLT to
get the other localization file structure.

So the question is, why would you choose XLIFF over ll-XML for your own
extensions? The answer will be easy once we implement the last bits of
what XLIFF really could bring us: support for plurals. And this is what
I want to discuss with you here before starting/coordinating the
implementation.

Why do we need plurals?

Think of one of your extension that searches some records and shows a
result list, you probably will want to show some (localized) text such as:

N records found

where N is some number, typically the output of a SELECT COUNT(*)
statement. How do you do that currently? Well, you define some text in
your XLIFF (or ll-XML):

<trans-unit id="recordsFound" xml:space="preserve">
	<source>%s records found</source>
	<target>%s enregistrements trouvés</target>
</trans-unit>

and then do something like

$totalRows = $GLOBALS['TYPO3_DB']->exec_SELECTcountRows(...);

return
sprintf($GLOBALS['LANG']->sL('LLL:EXT:yourext/path/to/xliff:recordsFound'),
$totalRows);

OK, so far so good, this allows your %s placeholder to be put at some
other place because in some languages it may not be prepended to the
text for instance, with something like that (translated of course):

Records found: %s

But we already face a problem here, if we want to be correct, we have to
first deal with a single result row and as such having the text

1 records found
or
1 enregistrements trouvés

is definitely wrong! So we can start up with two texts:

<trans-unit id="recordFound" xml:space="preserve">
	<source>%s record found</source>
	<target>%s enregistrement trouvé</target>
</trans-unit>
<trans-unit id="recordsFound" xml:space="preserve">
	<source>%s records found</source>
	<target>%s enregistrements trouvés</target>
</trans-unit>

And do the business logic ourselves:

if ($totalRows > 1) {
	return ... ':recordsFound' ...
} else {
	return ... ':recordFound' ...
}

Where the %s does not seem to make sense for the singular version but
one may think we will be able to use it for 0, aka "no result" as well
so it still makes sense but... bing! This is a typical case where
English and French differ because 0 is plural in English whereas it is
singular in French (and many other languages):

0 records found (<-- plural)
0 enregistrement trouvé (<-- singular)

And to be correct in other languages (Russian, ...) we should ideally
grab the concept of "many" when dealing with plurals. Something like
2-3-4 is plural form #1 and then it's another form of plural (which is
normalized for each and every language and may be fine-tuned as well in
Pootle, in the administration area).

This is what XLIFF allows us! It supports the concept of different forms
of a label, allowing us to define one single trans-unit "id" and then
multiple forms.

So what is missing?

1) Support in TYPO3 for the different versions of the translated labels
(being able to parse them and access them)
2) Integration of the business logic of how to choose a label according
to some "counter" => we need to be able to give the counter to the
"sL()" method in order for it to retrieve the correct text for us

and hopefully, take advantage of this further refactoring to add support
for what is currently missing natively:

3) Support for placeholders within labels

I already can store this in my label and sprintf() it in my own code:

Showing records %s to %s out of %s

But what if the first %s is logically at another position in a
translated label? So we need something like

Showing records {START} to {END} out of {TOTAL}

and then implement the replacement logic in our code. I'd like to have
this natively supported by TYPO3.

Summary:

- We need a better sL() method
- We need to possibly pass a "counter" to choose the correct alternative
of a label
- We would like to be able to do string replacements right away

We have basically two possibilities, either we "include" the counter
value in some chosen replacement marker (by convention) or we separate
it from the replacement strings.

First solution allows to reuse the counter within the string:

{COUNT} records found

and call our sL() method with

public function sL($input, $hsc = FALSE, array $replacements = array()) {
	$counter = isset($replacements['COUNT']) ?
intval($replacements['COUNT']) : 1;
	// ...
}

Or we do it like that:

public function sL($input, $hsc = FALSE, $count = 1, array $replacements
= array()) {
	// ...
}

Which implies that we would have to pass twice the counter value but
allows us to use any marker in our string.

We could of course define additional methods such as

sLwithGreatReplacement()

or, more logically (^^) rename sL() to getText() and have
getTextPlaceholder()...

So, in short, the purpose of this thread is to discuss what we want. I
hope I explained the whole enough for every interested person being able
to think about it and provide feedback.

Thanks!

[1] http://translation.typo3.org

-- 
Xavier Perseguers
Release Manager TYPO3 4.6

TYPO3 .... inspiring people to share!
Get involved: http://typo3.org