[TYPO3-project-4-3] 4.3.0 vs. 4.2.7

[Rupert Germann]

Hi Dan,

Dan Osipov wrote:
> The graphs show a patched 4.2.7 - is that 4.2.7 with backported cached
> treelist?

yes. 
 
> It's interesting that patched 4.3 is slightly higher, but not completely
> equal to 4.2.7

yes. and since there are 3 key changes between 4.3 and 4.2.7 and I made 2 of
them equal in both versions - there is only one change left as culprit.
btw: if I profile a 4.3 page in Zend Studio I see that php spends quite a
lot of time in the autoloader class...
 
> Were the tests done on just one page, or across the site - I remember
> Ingo suggested MySQL cache will kick in with ab requests, making them
> not completely accurate.

I used siege requesting 100 different pages randomly. But I don't want to
discuss the testing methods here.
Example: If someone constructs a new motor for cars he wants to get some
comparable data like horsepower, torque and the like before the chassis of
this car is even drawn. To get this data he mounts the motor to a testing
facility putting load on the device to see how it reacts in different
loading situations.  
I'd say that's exactly what I did.

And it turned out that the new engine is not yet able to deliver the power
that the old engine delivered.

Here the testing evironment(s) in detail:
Environment 1: 

2 TYPO3 installations, one with TYPO3 4.2.7dev, the other one with TYPO3
4.3.0 (latest trunk). 
(later I patched them both and named them 4.2.7 patched and 4.3.0patched in
the results)
Both use identical databases. Apache and MySQL restarted after each test
run. Tests run several times. Server settings for apache and mysql are
slightly optimized to allow more apache childs and make mysql use more RAM
for caching. eaccelerator activated (php memory-limit: 128MB). TYPO3 4.3
caching framework uses dbbackend.
Tests done on ubuntu 8.04-64bit on a quadcore machine using 8GB RAM.

Commandline:
siege -c 30 -t 5M -i -b -f urls.txt

That simulates 30 concurent users (-c 30) clicking randomly the urls (-i)
from urls.txt 
(other options: b=throughput benchmark, no delay on concurrent users, t 5M=
run for 5 minutes)
urls.txt contains 101 randomly choosen urls from one TYPO3 site. 44 of them
point to fully cached pages the rest to pages with an USER_INT plugin.
Since the both sites are identical they use the same cHashes and so I used
the same list on both installations by simply str_replacing the server
address.

I also made tests on slower hardware (not in the graphics):
Environment 2: 
vbox server with 1 CPU and 768MB RAM, running ubuntu 8.10, apache and mysql
with default settings, same identical databases, eaccelerator activated
(php memory-limit: 64MB)

the tendency in the results is exactly the same as on the faster hardware:
TYPO3 4.3 is ~ 25% slower than 4.2.7

I would be interested if one of you gets different results.

greets
rupert