[TYPO3-english] No template found after conversion to UTF8...

[Andrew Plank]

Dmitry Dulepov wrote:
> Hi!
> 
> Andrew Plank wrote:
>> 1. Duplicate the filesystem of the site to be cloned to the new server.
>> 2. Using phpMyAdmin, export the database. Convert the sql script to UTF8
>> from iso-8859-1 using iconv.
> 
> Well... Same wrong way, even after I told you where to search for the right way...
> 
>> $TYPO3_CONF_VARS['SYS']['setDBinit'] = "SET NAMES utf8;\nSET CHARACTER
>> SET utf8;\nSET SESSION character_set_server=utf8;";   
>>
>> Notice the extra "SET CHARACTER SET utf8" statement.
> 
> I noticed. This particular statement corrupted many tables already.
> 
>> HOWEVER... If you try to map a TV template with that extra setDBinit
>> statement added in step 9, it won't save it. But if it's not there,
>> you'll get the dreaded "No template found!" error in the FE!
> 
> Thanks to the statement above, which messes up character sets. You are not the first one who is hit with it.
> 
>> So this is a solution if you don't mind altering localconf.php before
>> and after each and every mapping change in TV.
>>
>> Dmitry... Can you think of a reason why this should be the case? Is it
>> by design? Or just incidental to the way in which the T3 DB mechanism
>> works, or is it a bug? It would be great to get your input. If it is
>> possible to create a workaround for this - perhaps as a bugfix in the
>> next TV?
> 
> Everything works correctly if you do it correctly. Please, do what I told you in the previous post. It takes several keystrokes in a browser search bar to find the right solution in our bug tracker. I really do not understand why you went with iconv again after you were told it is a wrong way :(
> 
> You go the wrong way, it does not work and you think it is a bug... :(
> 
> This post is not to offend you or anything like that. It is just disappointing to me when people consciously and deliberately make the same mistake twice.
> 

Hi Dmitry,

I suppose in retrospect, I should have mentioned that I'd tried several 
procedures I'd found on the bug tracker, in particular the method where 
the SQL files are patched, the DB analyzer run, then the DB converted to 
Binary and then UTF8.

With all the methods I'd tried, I couldn't get any of them to work; I 
either ended up with a whole bunch of PHP errors, "No template found!", 
"No pages on rootlevel" or somesuch error. It seems obvious now that I 
should have stated this as the first thing in the post.

However, I submit to your point that I deliberately persevered with my 
attempts to find my own solution. I also apologise for labelling the 
aparrently duplicitous behaviour of TV where DB charactersets are 
concerned as a bug, when it quite obviously isn't.

Now I must state again that all the existing documented methods (that 
I've been able to find) for converting a DB  from latin1 to UTF8 have 
not worked for me, even though I've repeatedly (and painstakingly) tried 
the steps detailed in each one. I was therefore in a situation where I 
had to think outside the box to come up with a solution.

Perseverance pays off. If you will indulge me, there follows an outline 
of the steps I took to solve my issue. I would still appreciate any 
feedback you may have on this process, as well as the feedback of any 
other T3 devs that have far more knowledge of the inner workings of T3 
than I do.

1. Make a local copy of the fileadmin, typo3conf and uploads folders 
from the server to be cloned.
2. Create a small shell script that loops through all php, html, js, css 
and txt files contained within these 3 folders, converting them all from 
iso-5589-1 to utf8. (It's quick and dirty but it does the job. I'll post 
it as a reply to this post)
3. Copy the typo3 and t3lib folders from the "old" server to the new. 
(Or just create symlinks to the shared source as I did. I killed two 
birds with one stone with this process by symlinking to a shared 4.2.1 
source, thereby upgrading the site from 4.1.6)
4. Copy the typo3conf folder (converted in step two) to the "new" server.
5. Create a typo3temp and typo3temp/llxml folder on the "new" server.
6. Using your control panel, create a new DB and DB user.
7. Edit the localconf.php file and set the DB credentials, and add the 
following lines before the closing brace (Notice the use of SET NAMES only):

$TYPO3_CONF_VARS['BE']['forceCharset'] = 'utf-8';
$TYPO3_CONF_VARS['SYS']['setDBinit'] = "SET NAMES utf8;";
$TYPO3_CONF_VARS['SYS']['UTF8filesystem'] = '1';
$TYPO3_CONF_VARS['GFX']['TTFLocaleConv'] = 'UTF-8';

8. Delete typo3conf/temp_* files.
9. In the front end, visit /typo3/install, and run the database 
analyzer. This will create all the required tables.
10. Create an admin user, and login to the BE. Re-import 
static_info_tables, otherwise you'll get PHP errors in step 12.
11. On the "old" site, export the entire pagetree to a t3d file.
12. On the new site (which is completely empty, though has all the 
extensions from the "old" site installed) import the t3d file made in 
the above step, making sure to force all UIDs.
13. Now FTP the converted fileadmin and uploads folders from step two to 
the new server, overwriting the existing files (which were put there by 
step 12)
14. Re-map TV templates, and clear all caches. If you use RealURL you'll 
need to modify your config.baseURL setting. Also, you may need to modify 
your config.metaCharset.
15. Hit the front end. Hit refresh. It works for me, anyhow.

It's a long and convoluted path to get there, and it takes about 20 
minutes, and if you miss a step things go wrong enough to have to start 
over again, but it works for me, and I haven't found anything yet that 
gets broken by this process. Can you think of something that might get 
broken? All comments welcome!

Kind regards,

Andrew Plank