[Typo3-dev] Re: The future of typo3

[Michael Zedeler]

I got a digest, so I have included my answers to everybody in this mail.

For starters: I didn't mention the nice features of typo3. I do think 
typo3 is nice. But I won't use it because the scope where it is useful 
to me is too limited.

>>Item 1: Outdated interaction paradigm
>>    
>>
>In general I agree with you that it's hard to provide backend like
>editing functions to the frontend. And although I still think that the
>separation of backend and frontend users is a good idea, it's worth
>being discussed.
>
I would like to hear why the separation of frontend and backend is such 
a good idea. Most of you on the developer list seems to stick to it, but 
I disagree. If you stick with the backend, you have a "split 
personality" website, where code reuse is cumbersome.. Moreover you 
support the "publish-view"-paradigm. What is the aim of Typo3? I 
strongly believe that the content-publishing-paradigm is dead and future 
web applications should be more flexible. Microsoft Frontpage is an 
example of such a content-publishing paradigm. You edit your home page, 
add various semi-dynamic elements (such as poll, guestbooks and search) 
and publish it. What is the difference between Typo3 and Frontpage if 
you don't support user interaction through the frontend? Obviously it is 
the fact that the application for editing and publishing is already on 
the net, and has more facilities that supports multiple authors, but the 
paradigm is still pretty much the same.

Please enlighten me -why is it such a good thing to separate the backend 
and the frontend?

>> * The code that handles permissions works very differently depending on
>>   what part you're logged on to.
>> * The backend uses fairly advanced form-building libraries that seem 
>>   impossible or very hard to reuse as part of the front end.
>>    
>>
>There has been some brainstorming and development to provide an easy way
>of frontend editing, form rendering for frontends, evaluation and so on.
>Daniel Thomas is currently working on an extension providing a better
>connection between backend end frontend editing (i.e. rendering of
>frontend forms by using TCA information etc.). 
>
More extensions aggravates the problems that I am addressing. All the 
code needed for front end editing is already there. It just needs to be 
*enabled* for front end editing. All the "bridges" between front end and 
back end users (front end user - to back end user mapping, backend login 
in front end and so on) are symptoms of poor design. Making new 
extensions reimplementing forms already available in the backend are 
symptoms of poor design.

>I have also discussed the existing models (feadminlib, Julle's frontend
>editing API) and ideas for a new approach with Christian Jul Jensen. We
>did not find the time yet to publish our thoughts but Daniel Thomas has
>started writing a RFC already and we'd be glad if you joined us with
>some fresh thoughts about it!
>
I believe that I am doing so now.

>>This paradigm is outdated and leads to grievances that should never have 
>>been there. Why? Because it leads to redundant code.
>>    
>>
>I agree. But it might make sense to have a separation of frontend and
>backend, sharing most of the same code e.g. for rendering forms,
>evaluation, database access etc.
>
"Separation" is just a flag. Put it wherever you want it. Do you need to 
limit a users access to what is called "the backend", set the flag to 0. 
Does he need access? Set it to 1.

>>If you want to make a front end plugin that receives data from a user, 
>>you'll basically have to start from scratch. This is absolutely crazy, 
>>when you keep in mind that the backend has a lot of nice form-building 
>>libraries. It leads to longer development time and more (possibly critical) errors.
>>    
>>
>That should be solved by a frontend forms / editing API.
>
Se my comments above regarding symptoms of poor design.

>>Another, much more fundamental problem is that many websites uses heavy 
>>interaction with all users, and does not comply with the very rigid frontend/backend-scheme. 
>>An example is wikis where everybody have access to edit everything. 
>>I know it is possible to do with typo3, but only by letting people straight into a 
>>backend that uses a fixed layout with little room for customization. 
>>    
>>
>
>In my opinion, TYPO3 is not meant to be a wiki but I guess a frontend
>plugin providing those functions should be easily made within a day or
>two. Dan Frost has been thinking about that already.
>
I know that typo3 is not intended to be a wiki, but I think that you can 
test the flexibility by comparing to existing systems. If the design is 
good, a mere reconfiguration should make it possible to emulate a wiki. 
And wikis are - of course - not the only worthy reference systems.

>>Another example would be any website using heavy user interaction with many 
>>levels of permissions (or roles), such as an information portal, an intranet, 
>>extranet or the like. 
>>In those cases, you have to either let the users straight into the backend when 
>>editing something or doing it all ground up.
>>    
>>
>That is a good point to discuss. I'd like to add more questions:
>
> - How about a new type of user which is allowed to create backend users
>but is not administrator himself (i.e. without access to TS templates)?
>
The backend/frontend idea needs to go away. I am not happy with it so 
its hard to answer this point.

> - What about a new user administration module which can easily handle
>1000+ (backend / frontend) users?
>
I don't know - why is this interesting?

>>Ask yourself: how many times have you developed a front end plugin that uses 
>>the rich text editor? My bet is none or very few because it is cumbersome. 
>>    
>>
>Should be much easier with the frontend API mentioned above.
>
Se my comments above regarding symptoms of poor design.

>>Item 2: Inconsistencies and loose ends in the system
>>Typo3 does not seem very consistent to me. Most things seem to have been invented 
>>and implemented without thinking about the overall design. I haven't had time to dig 
>>around in the code for long, so I'll just mention the conflict between typoscript and
>> the templates. 
>>    
>>
>We've already had a discussion about if TYPO3 is consistent or not. As
>I'm using TYPO3 for quite some time now, I don't see many
>inconsitencies, but I understand that it might look like there are many
>when you're getting started. The name "TypoScript" is a source of such
>ambiguities.
>
In my opinion: that is because that you have worked out a way of using 
typo3 already, and youøre not able to see how inconsistent the parts 
that you have gotten accustomed to avoid really are.

>>Typoscript (which is destined to be misunderstood by every newbee, since 
>>it is *not* scripting language) is able to produce and manipulate page content. 
>>    
>>
>Or better: TypoScript is for configuring the engine which produces page
>content output.
>
Interesting. There is an engine producing output. I haven't thought 
about it that way. Good point.

>>The same can be done using templates, so which one do you choose? 
>>Typoscript has access to templating functions, so why not choose it as a basis 
>>for almost everything you do? That is not possible, because typoscript isn't 
>>complete with regard to the tasks you may have to solve. Putting data in a table in
>>a database is one of those things not possible.
>>    
>>
>That's a point many people complain about: Often there is more than one
>way to solve a certain task. But behind all that there is TypoScript. 
>
>I hope I got you right: You mean HTML Templates vs. TypoScript
>Templates, right? Currently there are these two ways (plus PHP functions
>of course) of creating page content output.
>
I mean HTML-templates with what Kasper calls "markers" and "subparts".

>There are many people, who like the pure TypoScript way. They don't use
>external HTML files but put all their HTML tags into TypoScript objects
>and wraps. Usually that is the case if it's only one person creating the
>HTML code as well as the TYPO3 implementation.
>
And away goes the famous separation of code and layout.

>On the other hand there are people who prefer working with HTML files
>and use TypoScript to replace certain parts with content. That model is
>good for separating the work of a HTML designer and the TYPO3 developer.
>E.g. using the automaketemplate extension, the webdesigner simply
>provides an HTML file in a certain folder and won't have any contact to
>TYPO3 itself.
>

>So, what's the *better* way?
>
I must admit that I see HTML-templates as _the_ _only_ _way_ to do it, 
but if you say that many people like the typoscript way, I am ready to 
revise my standpoint regarding that.

>I think there's no answer to this and even worse: There are already
>ideas for a more powerful concept ...
>
Im all ears (eyes).

>>So what is the alternative? The alternative is to switch to PHP, which
>>has the effect that typoscript is hardly useful for any purpose besides
>>simple configuration of the application you write, and nothing else. 
>>My conclusion is that most serious work cannot be done without coding 
>>something in PHP, so why promote Typoscript so strongly? 
>>    
>>
>Because in the future it *will* be easier to accomplish many tasks which
>were not possible without PHP. TYPO3 3.6.0 will feature some new backend
>features, one of the is called FlexForms. With the help of some
>extension, it will be possible to create new content elements without
>using PHP.
>
Features are only worsening the problems that I have described, since 
they try to circumvent the symptoms arising from more fundamental design 
flaws.

>>Why is there almost no documentation of all the PHP-libraries for typo3, 
>>when this is actually what makes everything in typo3-world tick?
>>What I see when I look at the development plans is that there is a lot of 
>>effort put into developing new features. This is more fun for the developers, 
>>but please keep in mind that what I think is really neeeded is a big cleanup 
>>and a completely new set of documentation. I find it aggrevating to see little 
>>or no mention of theese problems on the website. Is it a "family secret"?
>>    
>>
>There's no secret about it. Kasper has spent most of his time with
>documentation of the source code and cleaning up. Have a look at the
>development version of TYPO3 3.6.0, look into the sources, everything is
>explained right there were you'll need it - in the source.
>
Sounds great. Will look.

>All comments are JavaDoc compatible, so it will be possible to create an
>automated documentation out of the source comments.
>
Another good thing(tm).

>Additionally some new documentations taking care of the fundamental
>concepts of TYPO3 development and a description of the API are on the
>way. What is still needed, are people helping us with these tasks.
>Anybody who's interested in offering help should have a look at the
>Teams and Projects page (http://typo3.org/1448.0.html)
>
I still think that several documentation projects are lacking. Se my 
original posting.

>>Item 3: Lacking architecture documentation
>>I have had a hard time trying to find a manual that explains the core of typo3. 
>>When developing typo3-plugins using PHP, I need some kind of roadmap that explaings 
>>how the most important parts of the system works.
>>What it should cover is a description of the request-response-cycle, how parameters
>>are passed to various parts of the system, user authentication and the properties 
>>and methods of the TFSE.
>>    
>>
>Partly on the way. However, if you have ideas on how to create such a
>documentation or have examples of good documenation already done, you're
>welcome to help us.
>
Sounds nice. I don't  think that I'll join in.

>>Item 4: Lacking best practise documentation
>>There is no documentation specifying how to write a plugin from a-z with 
>>all bells and whistles - using best practise. New developers are left to 
>>reinvent their own best practise which can be very time consuming.
>>    
>>
>We just started a reviewing proces of the existing extensions. The idea
>behind it is to spread the knowledge of how efficiently program TYPO3
>extensions and ensure a certain quality standard.
>
I think it is more appropiate to give such a document as "Getting 
started with typo3" an overhaul.

>However, you're right - a bunch of extension programming tutorials
>should be done. It's all about lacking time and manpower to do that
>quickly.
>
Fair enough. May I suggest that you try giving better and more explicit 
support to the pople writing documentation? You can do this by offering 
authors of books about typo3 th sell their books directly through 
typo3.org and typo3.com. Good documentation takes a really long time to 
write, and it will never be adequate unless people are paid to do it.

>>Summary:
>>I think that the future development of typo3 should be focused on the following items:
>>
>> * Get to grips with the actual intensions of the different parts of the
>> system. What is typoscript supposed to do? Are developers supposed to work mainly
>> in PHP? If "yes", please provide documentation! Are developers supposed to work 
>>mainly in typoscript? If "yes", please make it complete!
>> * Working out better documentation - see above.
>> * Cleanup of code with two main targets: (1) make it available for reuse everywhere - 
>> eliminate the separation between frontend and backend and (2) rethink local design
>>    
>>
>I'm glad that - except for the frontend / backend ideas - we have come to the same goals 
>for our future work. 
>
Nice to hear that you write that you agree, but is this actually what is 
taking place right now? It seems like there is a bunch of new "feature 
packs" on the way.

> Generally I agree to your comments. Most of the issues are already 
> adressed. Nevertheless comments like yours are highly appreciated.
>
Thanks. And thank you for taking time to reply. It has been frustrating 
to work my way into typo3. This helps.

> Software - not only Open Source - is some evolutionary thing. Some of 
> the things in TYPO3 would be made a little different by now if we 
> would start from scratch. On the other hand we still belive in the 
> overall concepts. For example, when extensions were introduced this 
> could have been done with minor changes. All the interfaces were 
> already there.
>
The principles behind the extensions are really good. What I would like 
to see is transferring more functionality into extensions and cutting 
down on the core functionality.

> Some of the issues you described depends on the seperation of frontend 
> and backend. I can not simply agree to your advice to use only one 
> user table, but that is not the point. The seperation of FE and BE is 
> intended. That means the BE is a database application which can run 
> without any FE.
>
Fair enough. But the back end should be an application written on 
exactly the same foundation and with the the same tools as everything in 
the front end. My point is not that I don't like the backend. What I 
don't like is that it is being treated as a completely separate 
application in every respect all the way down to the very core of typo3. 
Thats not nice.

> The FE don't have to be a website.
>
I'd like to get an example - not that I doubt what you write - I just 
don't really know what it could be.

> Of course you're right if you say BE and FE should share more 
> functionality. BTW: do you know that you can edit content in the frontend?
>
Well yes - I do it by writing my own php code in various frontend 
plugins. Please let me know if there are other ways.

> We'll try to move the codebase to be more abstract and reusable, which 
> will have the effect to share more functionality in BE and FE. We 
> belive that it can be done without a big compatibility break.
>
That sounds really nice.

> In order to complete the picture: I (the marketing and innocation guy) 
> support most of your points, quite along René's and Roberts line of 
> argumentation. Making TYPO3 more modular and thereby more less 
> redundant, partly unfinished and undecypherable, is what I consider 
> the biggest mid term challenge on the route from evolving from the "we 
> publish - they read"-application to the toolbox that we know we could 
> have. Thanks again for sharing your thoughts and I would be glad, if 
> there is a way to interest you in bearing with the evolution of TYPO3. 
> There is much good code and functionality to be opened up.
>
I'll be following typo3 and surely whish you good luck.

> - Some of your points are not fair when you talk about paradigms and 
> ways things should work which obviously is just your own idea. 
> Sometimes, what you call outdated, we call a feature. And customers agree.
>
I hope that someone will explain why the fe/be-separation is a good 
thing. I am opposing the idea until then.

> - Some of your points are good, I agree, but I have done my best, is 
> still doing and have in the process also become aware that my 
> ambitions for TYPO3 which could easily be exactly what you have 
> described is just not possible for me to do overnight. This is already 
> hard enough to accept, that your thoughts are always lightyears ahead 
> of what you can actually get done.
>
I know. But when I read some of the source code I keep thinking to my 
self: "what went through the developers head when he decided to tie this 
nice piece of code into the backend?". Much can be accomplished by 
avoiding interwoven globs of code that depend heavily on each other when 
developing new code.

> All in all I'm very proud that TYPO3 has become what it is. When I 
> think about the process of making this project I'm very proud to see 
> how FEW inconsistencies there are after all. I'm proud to realize how 
> often I made a decision that after all made TYPO3 better than another 
> decision would have meant for it.
>
Typo3 has clearly come a long way. The challenge now is real maturity, 
which requires a more explicit, formalised architecture and more 
documentation.

> TYPO3 is not perfect but I've tried my best to make it. And to the 
> degree it has been realized I'm actually exceedingly satisfied. And 
> honestly that makes me trust myself and my views far more than yours.
>
The saying "everbody prefers the smell of their own shit over the smell 
of others'" goes for developers too. Discussing *why* it smells better 
using logical arguments helps us adopt a common standard that many 
people will like.

> And lets say that everyone agrees with you and suddently nobody wants 
> to use TYPO3; I would still be happy with my creation since I works 
> for me and actually that was where it all started anyways; To create a 
> tool to help myself. I'm sorry for sharing that with the world. What a 
> sin.
>
Please don't be offended. I am writing this to help. It may sound harsh, 
but I _will_ respect arguments against the standpoints that I have 
presented. It is clear that typo3 is your darling (a darling that need 
not be killed!). But I think that too much "I know whats best because I 
just know" causes degeneration of the system. Its a good idea to start 
discussing why it is so great and what can be done (even) better.

> Finally michael; If you had had 5 years of full time development 
> starting also 5 years back in experience - would you claim that you 
> could have achieved something better than TYPO3? In terms of quality, 
> community, documentation, consistency, standards etc?
>
I have been busy making money and running companies.

> I'm sorry TYPO3 was not the system for you, there must be something 
> else outthere since there are so many things wrong.
>
Who knows. I am currently running a survey now and hope to find something.

> Thanks for your comments. It could have been more constructive if you 
> offered and proved your own help in terms of solving some of these 
> issues and maybe mentioned a few nice things about the system on top 
> of it - currently you have portraited yourself as a bitter man who has 
> spend a lot of time on a system which does nothing right.
>
Its hard to defend taking part in something if it is based on principles 
that you disagree with.

> So why bother giving us advice?
>
Because I think that it was the best thing that I *could* do for the 
project.

Regards,

Michael.