[TYPO3-core] RFC: Fix (part of) #7589: FULLTEXT keys are not processed correctly

[Michael Stucki]

Hi Martin,

> Michael Stucki schrieb:
>> This will be essential for full UTF8
>> support (which unfortunately did not make it into this version).
> 
> What is missing?

See my notes at http://bugs.typo3.org/view.php?id=6098

There are two different situations and I think it would maybe be ok to
enable this for all new installations:

- enforce UTF8 usage (using these patches, uncomment the line that ignores
  COLLATE fields)
- block the Install Tool if client, connection & result charset are not UTF8

What do you think?

However, it will be much more difficult to solve this for existing setups as
we have many different scenarios on how the current database charset is
used:

- DB is non-UTF8, Connection is non-UTF8, Content is non-UTF8
  => All fine, just set setDBinit="SET NAMES utf8" and enable the UTF8
     enforcement. All content will be converted correctly

- DB is non-UTF8, Connection is non-UTF8, Content is UTF8 (thanks to
  multiplyDBfieldSize)
  => If you convert this database the content will be twice-encoded UTF8
  => Solution is to convert from <current> to binary, and then from binary
     to UTF8 (skips the content conversion)

- DB is UTF8, Connection is UTF8, Content is UTF8
  => perfect no change is needed

- DB is UTF8, Connection is different, Content is UTF8, 
  => stupid, I would not support this :-)

So as you see we are pretty close at least in the case of new installations.
For existing setups, I would propose to write an article on how to migrate.
Good thing is that users would only have to change two lines in the code to
make the conversion succeed.

What do you think? If my patches get approved by today and beta3 would be
delayed until tomorrow, I would offer to take care of the missing steps:

- remove multiplyDBfieldSize from config_default.php (so the feature is
  still available, but it can no longer be set for new installations)

- instead of the commented out line, add a TYPO3_CONF_VAR to enable
  character set enforement when comparing at the Install Tool

- if this flag is enabled, block the Install Tool unless client, connection
  and result charset are UTF8

- and finally: enable the setting for new installations by modifying the
  1-2-3 installer and the localconf.php of the dummy-sites

Cheers, michael
-- 
Use a newsreader! Check out
http://typo3.org/community/mailing-lists/use-a-news-reader/