[Typo3] Xhtml compliance: problem with non_sgml chars

[Christopher]

Hi,

--- Peter Niederlag <niederlag at ikd01.de> wrote:
> Hi,
> 
> Christopher schrieb:
> [...]
> > This is sometimes caused by users pasting content from windows
> applications
> > into  the back end; you should be able to prevent it by forcing the BE
> > charset from the install tool. Look for :
> > 
> > [BE][forceCharset]
> > 
> > ....and set it to whatever's most appropriate (could be iso-8859-1,
> could be
> > utf-8 etc...)
> 
> Can somebody explain what is happening when this is set?
> 
> The content then somehow gets processed by TYPO3?
> Chars will beconverted?
> How?


Well superficially, the charset in the BE pages is set to the value you
choose (instead of the apparent default 'iso-8859-1'). The fact that the BE
and FE charsets can be different (as well as the various settings in the
install tool for charset conversion) seems to mean that content can be
converted on the way *out* of the database, and the fact that it's
necessary to change the charset of any already-entered content in the db
when the BE charset is changed makes me think that the BE charset value is
the charset used to store content in the DB - though I could be wrong :)


> Does this need any special care for mysqld as well. (Especially for 
> UTF-8)? Or is this all taken care of just by "[BE][forceCharset]"?
> 
> I do have problems with "non-SGML" as well right now and don't 
> understand the background yet. ;(
> Any hints appreciated. ;)

In my experience, the non-sgml error is usually caused by html entities [1]
that are part of non-standard Windows charsets (windows-1252 ?). For
instance, I've encountered this problem (using a different CMS) when users
copy and paste content from Windows apps such as notepad. Forcing the BE
charset in Typo3 seems to prevent the page failing validation, although
AFAICT the offending entities then get rendered as question-marks...


-Christopher

[1] http://www2.cddc.vt.edu/marxists/admin/charsets/non-sgml.htm

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com