[Typo3] entities & character is replaced by & on site

[Simon Tuck]

Søren Schaffstein wrote:
>>>>Looks to me like problem with the htmlSpecialChars property: You're
>>>
>>>bodytext is going through htmlSpecialChars twice. 
>>>
>>>
>>>>Probably once from the rte to the db and then from the db to the fe. 
>>>>try
>>>
>>>disabling htmlSpecialChars in your rte. Add the
>>>
>>>
>>>>following lines to your page tsconfig:
>>>
>>>
>>>>RTE.default.proc.entryHTMLparser_db = 1 
>>>>RTE.default.proc.entryHTMLparser_db.htmlSpecialChars = -1
>>>
>>>
>>>>cheers
>>>>simon
>>>
>>>
>>>Simon,
>>>
>>>Sounded like a good idea, but too bad it didn't work. I assume the 
>>>transformation happens from db to fe since in the db everything looks
> 
> fine.
> 
>>>Perhaps it would be a good idea to shut down the htmlSpecialChars 
>>>conversion from db to fe. But I couldn't find out how to do this.
>>>Can you help with this as well?
>>>
>>>Cheers,
>>>Soeren
>>>
>>
>>nope. that usually works for me. what have you got in your page tsconfig?
> 
> Please find the relevant parts of my tsconfig below - perhaps this helps...
> (I left the "htmlSpecialChars" line in code but commented it out.)
> Soeren
> 
> #set processing options for RTE
> RTE.default.proc {
>   #transformation method
>   overruleMode = ts_css
> 
>   #xhtml_cleaning = 1
>   dontConvBRtoParagraph = 1
> 
>   #tags allowed outside p & div
>   allowTagsOutside = img,hr
> 
>   allowTags = table, tbody, tr, th, td, h1, h2, h3, h4, h5, h6, div, p, br,
> span, ul, ol, li, pre, blockquote, strong, em, b, i, u, sub, sup, strike, a,
> img, nobr, hr, tt, q, cite, abbr, acronym
>   denyTags = font
>   allowedClasses = orange, orange-bold, gray, small
> 
>   #clean tags
>   #noAttrib = b, i, u, strike, sub, sup, strong, em, quote, blockquote,
> cite, tt, br, center
>   tags.hr.allowedAttribs = class
> 
>   #rmTagIfNoAttrib = span,div,font
> }
> 
> # parser content to rte
> HTMLparser_rte {
>   allowTags < RTE.default.proc.allowTags
>   denyTags  < RTE.default.proc.denyTags
>   allowedClasses < RTE.default.proc.allowedClasses
> }
> 
> # parser content to db
> RTE.default.proc.entryHTMLparser_db = 1
> RTE.default.proc.entryHTMLparser_db {
>   #try to disable special chars parser for polish
>   #htmlSpecialChars = -1
>   
>   allowTags < RTE.default.proc.allowTags
>   denyTags  < RTE.default.proc.denyTags
>   allowedClasses < RTE.default.proc.allowedClasses
> }
> 
> 
> # configuration of HTMLarea
> RTE.default {
>   hidePStyleItems = H6,PRE
>   contentCSS = fileadmin/template/css/inpage.css
>   showButtons = textstyle, formatblock, bold, italic, underline, subscript,
> superscript, left, center, right, orderedlist, unorderedlist, outdent,
> indent, textindicator, insertcharacter, link, image, table, findreplace,
> spellcheck, removeformat, copy, cut, paste, undo, redo, about,
> toggleborders, tableproperties, rowproperties, rowinsertabove,
> rowinsertunder, rowdelete, rowsplit, columninsertbefore, columninsertafter,
> columndelete, columnsplit, cellproperties, cellinsertbefore,
> columninsertafter, celldelete, cellsplit, cellmerge, chMode
>   hideButtons = textstylelabel, blockstylelabel, blockstyle, strikethrough,
> fontstyle, fontsize, lefttoright, righttoleft, justifyfull, textcolor,
> bgcolor, emoticon, line, user, acronym, inserttag, showhelp
> # normally add chMode to hidden buttons!
>   toolbarOrder = blockstylelabel, blockstyle, space, textstylelabel,
> textstyle, linebreak, fontstyle, space, fontsize, space, formatblock, bar,
> bold, italic, underline, bar, strikethrough, subscript, superscript, bar,
> lefttoright, righttoleft, bar, left, center, right, justifyfull, bar,
> orderedlist, unorderedlist, outdent, indent, bar, textcolor, bgcolor,
> textindicator, bar, emoticon, insertcharacter, line, link, image, table,
> user, acronym, bar, findreplace, spellcheck, bar, chMode, inserttag,
> removeformat, bar, copy, cut, paste, bar, undo, redo, bar, showhelp, about,
> linebreak,  toggleborders, bar, tableproperties, bar, rowproperties,
> rowinsertabove, rowinsertunder, rowdelete, rowsplit, bar,
> columninsertbefore, columninsertafter, columndelete, columnsplit, bar,
> cellproperties, cellinsertbefore, columninsertafter, celldelete, cellsplit,
> cellmerge
>   showStatusBar = 1
>   # css classes for images
>   classesImage = 
> }
> 
> #list of allowed classes on tags (neccessary to preserve span tags)
> RTE.config.tt_content.bodytext.proc.allowedClasses <
> RTE.default.proc.allowedClasses
> 
Hiya,
sorry for my belated reply. looks pretty similar to my own config - i can see you're using both the htmlarea and the 
standard rte. i can offer you my rte configuration. perhaps that helps... :)
cheers
simon

/*
PAGE TSCONFIG ------------------------------------------------------
*/

[...]

RTE >
RTE.classes {
	links {
		name = links
		value = float: left; margin: 5px; margin-left: 0px;
	}
	rechts {
		name = rechts
		value = float: right; margin: 5px; margin-right: 0px;
	}
	zentriert {
		name = zentriert
		value = margin: 5px; display: block; text-align: center;
	}
	h2 {
		name = Heading 2 (H2)
		value = margin: 0; font-size: 12px; font-weight: bold; text-decoration: underline;
	}
	durchgestrichen {
		name = durchgestrichen
		value = text-decoration: line-through;
	}
	kursiv {
		name = kursiv
		value = font-style: italic;
	}
	normal {
		name = normal
		value = font-weight: normal;
	}
	unterstrichen {
		name = unterstrichen
		value = text-decoration: underline;
	}
}

RTE.default {
	## IE RTE Configuration
	mainStyleOverride (
		body {
			border: 1px black solid;
			border-top: none;
			margin: 0px;
			color: #000000;
			background-color: #FAC9CB;
			font-family: Arial, Helvetica, sans-serif;
			font-size: 12px
		}
		td {background-color: #FAC9CB;font-family: Arial, Helvetica, sans-serif;font-size: 12px;margin: 0;padding: 0;font-weight: 
bold}
		p {background-color: #FAC9CB;font-family: Arial, Helvetica, sans-serif;font-size: 12px;margin: 0;padding: 0;font-weight: bold}
		p:after { clear: both; content: "."; display: block; height: 0; visibility: hidden;}
		p { display: inline-table; }
		/* Hides from IE-mac \*/
		* html p {	height: 1%; }
		p{ display: block; }
		/* End hide from IE-mac */
		h2 {margin: 0; font-size: 12px; font-weight: bold; text-decoration: underline;}
		div {margin-top:0px; margin-bottom:5px;}
		ol {margin: 5px 10px 5px;}
		ul {margin: 5px 10px 5px;}
		blockquote {margin-top:0px; margin-bottom:0px;}
	)
	
	## HTML AREA Configuration
	skin = EXT:rtehtmlarea/htmlarea/skins/default/htmlarea.css
	contentCSS = fileadmin/templates/css/text-normal.css
	enableWordClean = 1
	removeComments = 1
	removeTags = sdfield
	removeTagsAndContents = style,script
	useCSS = 1
	defaultLinkTarget =
	showStatusBar = 1
	showButtons = 
formatblock,class,copy,cut,paste,undo,redo,textstylelabel,textstyle,image,link,insertcharacter,chMode,removeformat,findreplace
	toolbarOrder = 
formatblock,bar,textstylelabel,textstyle,linebreak,link,bar,copy,cut,paste,bar,undo,redo,bar,image,insertcharacter,bar,chMode,removeformat,bar,findreplace,bar
	hideButtons = 
showhelp,about,textindicator,fontstyle,fontsize,textcolor,bold,italic,underline,left,center,right,orderedlist,unorderedlist,outdent,indent,table,bgcolor,emoticon,line,user
	hideTableOperationsInToolbar = 1
	keepButtonGroupTogether = 0
	disableContextMenu = 0
	disableSelectColor = 1
	disableTYPO3Browsers = 0
	disableEnterParagraphs = 0
	removeTrailingBR = 1
	hidePStyleItems = H1, H3, H4, H5, H6, PRE, ADDRESS
	hideFontSizes = *
	hideTags = font, font (full)
	disableColorPicker = 1
	classesCharacter = normal,kursiv,unterstrichen,durchgestrichen
	classesParagraph =
	classesImage = rechts,links,zentriert
	classesAnchor =
	showTagFreeClasses = 0

	## Default proc rules
	proc {

		## TRANSFORMATION METHOD
		overruleMode = ts_css

		## LINES CONVERSION
		dontConvBRtoParagraph = 0

		## BRING FONT TAGS INSIDE PARAGRAPHS
		internalizeFontTags = 1

		## TAGS ALLOWED OUTSIDE P & DIV
		allowTagsOutside = img

		## TAGS ALLOWED
		allowTags = p,br,a,link,li,ul,ol,span,img,h2

		## TAGS DENIED
		denyTags = table,tr,td,b,i,u,div,center,pre,font,hr,sub,sup,strong,em,li,ul,ol,blockquote,strike

		## ALLOWED P & DIV ATTRIBUTES
		keepPDIVattribs = class
		//preserveTags = p,div
		
		## ALLOWED CLASSES
		allowedClasses = normal,kursiv,unterstrichen,durchgestrichen,rechts,links,zentriert
		
		# Forces DIV sections to P
		//remapParagraphTag = p
		
		## CONTENT TO DATABASE
		entryHTMLparser_db = 1
		entryHTMLparser_db {

			## TAGS ALLOWED
			allowTags < RTE.default.proc.allowTags
			allowTagsOutside < RTE.default.proc.allowTagsOutside
			allowTagsInTypolists = br,a,link,span,img
			denyTags < RTE.default.proc.denyTags

			## XHTML COMPLIANCE
			xhtml_cleaning = 1

			## AVOID TO HSC TWICE
			htmlSpecialChars = -1

			## CLEAN TAGS
			tags.hr.allowedAttribs = class
			tags.span.allowedAttribs = class
			tags.p.allowedAttribs = class
			noAttrib = b,i,u,strike,sub,sup,strong,em,quote,blockquote,cite,tt,br,center,p,h2

			## REMAP B AND I TAGS
			//tags.b.remap = strong
			//tags.i.remap = em

			## REMOVE OPEN OFFICE META TAGS
			removeTags = sdfield

			## PROTECT CUSTOM TAGS
			keepNonMatchedTags = protect
		}
	}
}

## front end RTE configuration
RTE.default.FE < RTE.default
RTE.default.FE.showStatusBar = 0
RTE.default.FE.hideButtons = chMode

## tt_content RTE configuration
RTE.config.tt_content.bodytext < RTE.default
//RTE.config.tt_content.bodytext.showButtons = *

## tt_content TCEFORM configuration
TCEFORM.tt_content.bodytext.RTEfullScreenWidth= 100%

[...]

/*
TEMPLATE SETUP ------------------------------------------------------
*/

[...]

### RTE RELATED ###
tt_content.text.20.parseFunc.tags.typolist.breakoutTypoTagContent = 1
tt_content.text.20.parseFunc.tags.typohead.breakoutTypoTagContent = 1
tt_content.text.20.parseFunc.nonTypoTagStdWrap.encapsLines {
	encapsTagList = div,p
	addAttributes.P.style =
	addAttributes.PRE.style =
	addAttributes.A.class =
	addAttributes.P.class =
	nonWrappedTag = p
	wrapNonWrappedLines = |
	innerStdWrap_all.ifEmpty = &nbsp;
	innerStdWrap_all.textStyle < tt_content.text.20.textStyle
}
tt_content.text.20.textStyle >
lib.stdheader.10.stdWrap.wrap = |
tt_content.header.20.stdWrap.wrap = |
lib.parseFunc_RTE.nonTypoTagStdWrap.encapsLines.addAttributes.P.class =

[...]