[TYPO3-v4] Core performance for 4.6 and beyond

[Christian Kuhn]

Hey.

I've recently done some profiling of the TYPO3 core with 4.5 in 
comparison to 4.4 and would like to present my findings and conclusions.

I'm talking about PHP parsetime in the FE at this point, not about user 
experience and JS framework speed.

Basic numbers:
* From my experience 4.4 and 4.5 for uncached pages show pretty much the 
same speed. We've done some patches to speed up some widely used 
t3lib_div methods, but some of them where ported to 4.4 as well, so the 
overall speed difference is not significant. Further improvements can be 
done if someone digs into details and speeds up often called or 
expensive methods somehow.

* 4.4 vs. 4.5 for fully cached pages: 4.5 is actually about 10-15% 
slower than 4.4. We're talking about some milliseconds here. Reasons for 
the slowdown: First and foremost the preparedStatement stuff was added 
to the core, which is always loaded for fully cached pages. The class 
loading, instantiation and stuff takes about 5-10% additional time. I 
know that we want and need this, so we have to live with the performance 
impact for now. Secondly (speaking about the introduction package) 
extbase added another cache, it's instantiation takes 5-10% as well 
(while it might boost other calculations).


About full cached pages:
* A stripped down site without too much functionality (eg. conditions) 
can be delivered within 10ms on a decent server. This sums up to an 
impressive number of requests for a standard multi core system. Taking 
the yearly CPU performance gain into account it is more easy to throw 
more hardware to the problem than trying to squeeze even more 
performance out of TYPO3 'full cached'. If more speed is needed, other 
solutions on other levels like varnish & friends are much more effective 
anyway.

* The base performance of fully cached pages is not that bad at all. 
Improving this significantly requires major refactoring (for example 
things like performance optimized fe_user determination, and quicker 
match conditions).

* Class (file) loading and instantiation takes 10-20% of the render 
time. Time is consumed by makeInstance and other classes which trigger 
methods that use the autoloader. Every additional class *will* add 
parsetime for full cached pages, consumed time *is* significant for each 
and every additional class that must be loaded.

* We *will* split more and more classes to smaller packages, we *will* 
refactor code to make it better maintainable, we *will* write more 
smaller methods. All that will *add* parsetime in the first place.

* In real live use-cases with big caches other things get more important 
than the rough parsetime of PHP: For example mysql slows down with big 
tables.


Conclusion:
* Full cached pages are already beyond maximum performance, we will slow 
down with further versions because of added functionality and more and 
smaller classes. Parsetime of autoloading related things will be more 
and more expensive from version to version.

* The only chance to gain speed in the long run is if we do significant 
refactoring of current caching mechanisms, especially during boostrap.


What could be done from my POV:
* First and foremost the 'old caching' must be dropped: The caching 
framework is *much* more flexible and gives tons of options for specific 
needs. It should be instantiated earlier to make use of it very early in 
the bootstrap process (eg. in the autoloader). This will actually drain 
performance in the first place but will speed up a lot if we are able to 
refactor other things.

* We must refactor TYPO3 boostrap to make it more flexible for caching: 
We should be able to interweave caching at every level we want to get 
significant impact. Ideas:
** Push all autoloader arrays to *one* cache file using the php frontend 
(I have a working patch for that).
** Get rid of temp_CACHED* files and push to PhpFrontend in the first 
place, change logic completely in the long run (I have rough code for 
that, but there are several php scope problems).
** We could concatentate important (read: always needed) classes in one 
file to reduce filesystem lookup overhead. Making makeInstance aware of 
that shouldn't be too difficult (not tried yet). This will *add* 
performance if we refactor important classes to smaller things from 
which not everything is needed for full cached access.
** We should be able to cache and load global arrays more quickly. The 
concept of ext_tables.php and ext_localconf.php is a problem, caching 
should actually step in at some other level than just on concatenated 
ext_tables / ext_localconf files.


So much for now. Please read on in my second post "TYPO3 boostrap for 
4.6 and beyond"


Looking forward to your input and opinions,
Christian