[TYPO3-dev] FLOW3 / TYPO3 5.0 (was: blog_example gives error "A cache with identifier "cache_extbase_reflection" does not exist")

[Robert Lemke]

Hi folks,

sorry for entering the discussion at this late point in time - I first
needed some hint to find this thread.

I think that many of you raised valid questions in this thread so far
and also made valuable conclusions from what you know. Naturally, I
can't be totally objective about FLOW3 / TYPO3 Phoenix development, but
I might have some deeper insight to the topic which I'd like to share
with you.

As you know, we didn't start TYPO3 Phoenix / FLOW3 development as a fun
project. Back then we saw two big flaws in the current TYPO3 CMS:

  - edgy user experience / user interface
  - hard to change and overly complex code structure

With TYPO3 4.5 (which was just a version number far enough in the
future) we wanted to address the UX part. And I am glad that we see
quite some progress in that field. TYPO3 5.0 has the tag line "usability
for the developers".

Changing something in TYPO3 - like adding workspaces, versioning, a DAM
or concepts like Dependency Injection - is very hard work, at times
impossible to do. We don't have a clear public API or a reasonable unit
test coverage which could act as a safety net for bigger refactorings.
Even extensions like TemplaVoila or RealURL suffer from this: Instead of
having teams working on it, it can be only a certain person who has
enough overview to not break anything. The reason is poor design and a
non-agile development mode which doesn't match the reality in our
development community.

This cries for a new architecture and a new _way_ to develop as well. We
just can't proceed like that forever. That's what most of us agree on.

When we started with Phoenix development we realized that probably 80%
of TYPO3 is infrastructure code: user administration, templating,
caching, security, extension / package management, file handling,
resource management (ie. upload / download of files), configuration,
logging, web services, database handling - you name it.

So we started with this part, because, working with TYPO3 for 6 or 7
years, we _knew_ that we wanted to have certain infrastructure features.
If we really wanted to simplify plugin development, we needed a clear
MVC framework which takes care of the whole security related to user
input. If we want versioning, workspaces and flexible data structures as
used by TemplaVoila, we needed to be creative about storing that in a
relational database - and use that concept from the beginning.

Well, that's all no big news. Ideally we would have had small sprints,
adding feature by feature and release as early as possible. However, as
we decided to start with the 80% infrastructure part, there were no
really visible results - apart from the code examples. We were not happy
with this situation either. Developers need motivation and the best
motivation they can get is finished, released, visible results. But
eventually we created the infrastructure part as one big chunk –
something which we would have to do anyway sooner or later.

We didn't plan to write our own framework. In the beginning we used as
many third-party components as we could: We started with using
Jackrabbit as the content repository, used Zend Lucene, the Horde YAML
parser and some ezComponents and more. Originally we planned to use many
more parts of Zend Framework but we realized that the quality was not
always sufficient (in case of Zend Lucence for example). We signed the
Zend CLA, posted bugfixes, opened tickets, but we didn't get any
reaction. At that point we sat together and came to the conclusion: It's
a very bady idea to let 80% of your product depend on a third-party
framework you dont' have any control about. Especially if you have
demands on the framework which it delivers only by 50% or so.

This was 4 years ago and ZF has evolved since then. But still I don't
see a way how we could have introduced DDD, DI, AOP or even proper PHP
5.3 support including namespaces to ZF. Starting today, I would build
something on top of Symfony - or maybe fork Symfony. But we have made so
much progress that it doesn't make sense to start over.

And why do we need all this fancy buzz-word stuff? Isn't a bit of MVC
and using Smarty for templates enough?

Our mission is to build a "future-proof" CMS with an "enterprise-ready"
extension API. This means that we must simplify development of complex
applications for professionals. You don't learn OOP or test-driven
development over night, that's true. But for the professional developers
that we are (in the community), it's worth the hassle. From the feedback
we got so far, it turned out that many PHP developers were really
inspired by reading the books we recommended and experienced a real
boost for their work as a PHP developer.

I guess a mistake we made was giving the framework a name too early.
People tend to think that TYPO3 Phoenix and FLOW3 are distinct projects.
"We don't need FLOW3, we want TYPO3 5.0". But FLOW3 is 80% of TYPO3.

FLOW3 is not hard to learn. In fact it's easier to learn for complete
newbies than for experienced PHP developers. This is a principal
problem: OOP is hard to learn for people who are used to create
procedural code. The DDD approach is very intuitive and the naming and
structure conventions are so consistent like in no second framework I
know. FLOW3 does make use of a different way to develop, an agile way.
And for sure, that takes quite some effort to get used to. But this is a
decision we took - we want the TYPO3 community to learn and be
competitive for today's and tomorrow's development practice. Although
this means that we'll leave some people behind, it's our chance to stay
on the market in the future.

FLOW3 is not slow. I honestly hate it (although I seldomly use that
strong words) if people talk about speed or performance in general
without doing real benchmarks. Fabien tried to compare a few frameworks
with Symfony 2.0 but the tests are meaningless because in reality it
were hello world tests. FLOW3 was not configured correctly - while the
other frameworks had database access and sessions turned off, that FLOW3
setup was using them. Fluid templates were uncached and many more parts
were. In my opinion you cannot compare frameworks by turning all caches
off and see which one is faster. The caches are there for a reason and
FLOW3 has a very thought-through caching mechanism.

Have you read the FLOW3 1.0.0 alpha 8 release notes?
http://flow3.typo3.org/download/release-notes/100-alpha-8/
In that version we introduced the Static Object Container. It produces
all the configured dependencies into hardcoded PHP code (see
http://flow3.typo3.org/fileadmin/Uploads/StaticObjectContainerSample.phps).

This is all done automatically and it is extremely efficient. Now, the
point is that this was a speed optimization which was added afterwards
without having to change any API function. A real speed boost which
doesn't require any client code to be changed. This is only possible
because we have designed a supporting architecture. And it means that
the future speed optimizations which we have planned can be implemented
without much hassle.

If your framework intelligently produces static html files even for
content which is only partly cached - and if it does that transparently,
without your further ado - what can be faster? And does it then matter
that the framework itself takes 3 seconds to load on the very first hit
after deploying your application?

As I wrote: It's more important that we have a clean architecture and
let developers produce clean code in no time than having a framework
which is fast internally. Because doing the latter is only truly
possible if you have the first. That doesn't mean though that we don't
want to make FLOW3 core as fast as possible.

Now you know something about the reasons why we are developing FLOW3.
Left is the question when TYPO3 5.0 will be ready.

Dmitry, you propose that we should set a deadline, 1 year from now and
until then TYPO3 5.0 should be released. That doesn't make sense at all.
What's your definition for TYPO3 5.0 being done? What's the feature set?
Setting a deadline for something 1 year ahead is pure speculation and
against all rules of agile development.

We will have our first release of TYPO3 Phoenix on Friday 14th of May.
And the second release will be on June 11th. It'll have a small visible
feature set which grows from sprint to sprint. You are highly encouraged
to follow our sprints at http://forge.typo3.org/projects/show/typo3-v5 !

I can imagine that, as soon as we release the first sprint release,
people will grumble "so that's 5.0?". That's why we'll label the
releases "TYPO3 Phoenix Sprint Release 1, 2, 3..." etc. However, this
will be a truly usable solution after the first few releases. But it
won't have the visible feature set yet you might hope 5.0 will have.

I expect that it'll be hard to go through that first period. But it's
like the following: Below you see screenshots of two different websites.
The one has taken about 3 years to produce, the other one was made
within 3 hours, can you tell which is which?

http://twitpic.com/1ilksf
http://twitpic.com/1ilkuj

Hint: The first screenshot is a static HTML file, the other is based on
FLOW3.

Finally I'd like to point out Daniel Pötzinger from AOE who has -
without much fuss - offered his help to implement Scrum for TYPO3
Phoenix. He organized a meeting here in Lübeck and since then supports
us during the Daily Scrum meetings via Skype. Likewise Ben
van't Ende is a great help acting as a Scrum Master for our Phoenix sprints.

So, my summary for now (before the coffee) is: Guys, I appreciate your
concerns and I understand that you're impatient. We have achieved a lot
and it wasn't for nothing. TYPO3 Phoenix will be truly unique and excel
any other Open Source CMS. And visible results are on the road and will
be coming more frequently soon.

Honestly, I wouldn't have given up my job as a TYPO3 consultant, now
earning only a fraction, if I wasn't truly convinced that TYPO3 Phoenix
will be a success.

Keep up the spirit,

Robert