[Typo3-dev] Suggestions on TypoScript

[Arne Skjaerholt]

#include "std/disclaimer.h"
Here be dragons. Long and (probably) controversial mail ahead.

Hello all,
I've been doing some learning about TypoScript (which I'll abbreviate TS
for my own sanity from now on), and I've come to the conclusion that, on
the whole, TS is a rather shoddy way to do things. This may of course
not be a very welcome comment, but that's how I see it. I do however
also have quite a few suggestions of changes that (in my entirely
subjective opinion) would make TS better. Of course, more or less all of
these suggestions would completely and utterly break backwards
compatibility, but it might be something to consider for Typo3 4.0 (or
whatever the next major revision will be called).
For my views on TS I've relied quite a bit on the document "TypoScript
Syntax and In-Depth Study" (from now on, the Document) from the
documentation section on typo3.org, but if I'm mistaken about something,
please point it out.
Ordered more or less in accordance with their logical connections, my
suggestions:

1) Make the internal representation of the TS be what it looks like, an
object tree.
>From the user's perspective, the parsed TS is somewhat like a DOM tree,
with lots of nodes connected to a root node (which is implicit in the
case of TS). However the internal representation of the parsed TS is a
PHP array, which is a bit unfortunate IMO.
This will fix another thing which bothers me. Objects can, in addition
to member variables and attributes, have values proper. Snagged from the
Document:
backgroundColor = blue
backgroundColor.transparency = 95%

I feel that this would be better written as follows:
background.color = blue
background.transparency = 95%

Opinions may differ on this point, but I feel the latter syntax makes
more sense than the former. This is especially because object types are
assigned by doing what (to me) looks like object instantiation.

2) Error reporting, or the lack thereof.
OK, let's face it, Typo3 doesn't even attempt to tell the user if
something unexpected happens when working with TS. For example if the
user specifies a non-existant file in the TS, this is silently ignored.
This is, on the whole I find, somewhat of a bad thing. Also, assigning
no nonsensical attributes in the TS "object tree" is not a problem
because of the internal representation of the tree. Implementing
suggestion 1 would alleviate this by making assignment to invalid object
members create a PHP exception. All Typo3 has to do then is to catch the
error and present it to the user.

3) On disambiguation.
I find the assignmenty of string values in TS rather disconcerting,
inparticular the following snippet made my stomach turn:
select.where = colPos = 0
this is not exactly readble. If you're used to a language where the
assignment operator is right-associative (like, say perl, PHP, C or C++)
you'd expect this to assign the value 0 to the
variable/attribute/whatever colPos and after that to select.where
instead of what it really does, which is assign the string "colPos = 0"
to select.where. I suggest doing what most other languages do, string
data has to be surrounded by quotes (the exact position on single vs.
double quotes can be discussed, but some form of quotes would be the
Right Thing to do).

4) Comment weirdness.
The way block comments works in TS, is somewhat strange. That a block
comment has to extend to the end of the line where the close comment
marker is, is completely counter-intuitive to me. Also, I feel that the
single line comments should be changed as well. # is fine, but having
comments induced by a single / is weird, especially when the tutorial
docs more or less consequently use the C++ style with two slashes.
Partial duplication of something is rarely a grand success (this goes
for the next bit on conditionals as well).
This is, of course, not all that big a problem in itself, but I suspect
that it is symptomatic of a somewhat badly implemented language parser.
I haven't had the time to really dive into the guts of the TS parser
yet, but from what I've seen it seems to deserve a little work.
Seeing how it'll probably be written in native PHP (writing a C module
and loading that would've been a better idea performance-wise, but
getting the module loaded would be a pain in secure mode), some corners
will have to be cut, but I think implementing the language parser as a
two step process would be a good idea. First some sort of state machine
(a slightly modified Finite State Automaton or Finite State Transducer
springs to my mind) would tokenize the the TS into "lines" and "tokens".
Some errors, like badly balanced quotes/parens/braces would be caught in
this step.
Then the interpreter proper would loop over the tokenized TS and
actually do stuff. I'd implement this as a simliar algorithm as the
tokenizer, but with more eval-like calls so that we can actually create
the TS parse-tree.

5) TRUE, FALSE and conditionals.
Conditionals has to be one of the ugliest features in TS. TS in itself
has no real concept of TRUE/FALSE. All it knows is that a conditional is
to be passed to a user-specified class which determines TRUE/FALSE. I
find that this is a really hairy way of handling conditionals. What
constitutes TRUE/FALSE should be inherent in TS, as should the
comparison operators.
Further, the conditional syntax could use some work. It is (I think)
inspired by shell style conditionals, but has been perverted (perverted
in the linguistic sense, not as a pejorative) into something rather
unwieldy.I'd suggest getting closer to the shell style (the -entire-
conditional goes between one pair of brackets) or doing something
completely different.
Also, the [GLOBAL] and [ELSE] markers are far too similar to regular
conditionals. This should probably be fixed as well. [ELSE] is also
somwhat wonky as else if () (as we know and love from PHP, C and C++) is
rather unwieldy. This particular construct would have to be embedded as
a separate if() block inside the else block. Not impossible, but rather
ugly.
Precedence is another tricky point. Without the possibility of using
parens to enfore precedence different from the usual precedence rules of
the logic operators you seem to be rather stuck (I may be wrong on this
point, but from my understanding of the Document this is an
impossibility).

6) Assignments.
The variant assignments should have an added =, just to emphasise that
they're assignments. It might seem obvious that () and {} are
assignments, but when reading large amounts of code, additional visual
hints are usually help towards making the code more readable. Of course,
the () style would be obsolete if point 3) on quotes is implemented, but
the {} variant is rather useful, if not a bit prone to getting lost in
the rest of the code.

7) Funny characters?
I'd suggest doing what PHP and perl (and partially in shell) has done
with their variable names. Variable names are preceded by a funny
character (more or less techincal name borrowed from perl). In the case
of PHP and shell this is always $, in perl one of the characters in the
set [$@%*&]. Basically this would open for a more readable syntax on the
whole. It would make implementation of "reserved words" for various
tasks (unsetting for example) rather simple to replace the somewhat
ad-hoc syntax we have today.
Also, this'd make adding new cObject types rather cleaner. Saying $myObj
= CONTENT would be distinctly different from $myObj = $CONTENT (the
former would make $myObj an object of the type CONTENT, the latter would
copy the contents of $CONTENT into $myObj). Aliasing could be
implemented by borrowing the reference style of PHP or perl (that is,
either $myObj = &$CONTENT or $myObj = \$CONTENT).

8) Some comments on the Document.
Section 1.3.1: The sentence "Footballs sing red" is not syntactically
valid English. The word red is an adjective, and thus it cannot
characterize a verb, only nominal phrases. A valid example would be
achieved by replacing red with an adverb like quickly, or simply making
the phrase say "Footballs sing" which is syntactically valid English,
but perfectly nonsensical on the semantic level.

Hopfully you'll all take this as it's intended: as constructive
criticism. The current incarnation of TS may be a bit wonky, but it
could be something quite nice, which is what I'm hoping will come out of
this. A better TS.

Arne
:wq