[TYPO3-core] Taming the Performance in 6.2
tmaroschik at dfau.de
Wed Nov 6 14:51:59 CET 2013
Am 05.11.13 16:02, schrieb Ernesto Baschny:
> Package Management
> Since beta1 together with the Package Management we introduced a new
> Caching Backend for loading PHP classes and it's class aliases. The new
> concepts minimizes the amount of memory required by not keeping one huge
> array in memory, but it seems that has still some performance drawbacks
> due to heavy use of "require_once". Since this affects all hits (even in
> frontend with all cached), tackling this area will most probably bring
> us the biggest benefit.
With the introduced package management several concepts have been
introduced/reworked along that had to be reflected in the class loader.
If you ask yourself where do all that require files result from read on:
"Arbitrary length vendor and package namespace"
So now you can use YourCompany\YourSolution\YourWhatever as base
namespace. But there can also be another package present that uses
YourCompany\YourSolution as base namespace. This is why you cannot
derive the package namespace deterministically from the class name.
Composer and Flow solves that by sorting the base namespaces (called
prefixes in composer) by length and checking for a match. See  for
code. This means for the worst case, in a system with 60 packages the
class name to load is been checked up to 59 unmatching prefixes. A
typical frontend requests loads about 200 classes. This means at worst
200 * 60 = 12000 strpos calls to resolve the package path of the class
name. This is still done now but a match is saved in the caching
framework resulting in a "proxy require" file.
"Keeping nothing more in memory"
With the introduction of namespaces the class alias mechanism has been
established. Due to the renaming of all class names into more meaningful
ones a map of about 1350 old to new class names had to be kept in
memory, because if the old name entered the class loader first the class
loader has to know that this class name is an old one. The inverse map
had to be kept too, because if the new name entered the class loader
first the aliases for the old ones have to be established. This results
in loading two array with 1350 entries upon bootstrap. Further the
ext_autoload.php files of extensions have been merged and loaded too,
which results in another potentially large array. The upfront loading
took constant long time upon every request. The process changed in the
new implementation so that a cache entry for each ext_autoload entry and
class alias is set up resulting in a "proxy require". After the first
request there is no need to load the 3 arrays anymore and logic to
retrieve the class file from the original and aliased class name is
"Resolving concurrent write issues"
The old class loader kept a map of class names to class file paths in
memory. Upon request shutdown when any changes to that map happened. it
has been simply serialized to a cache entry. If two request run in
parallel and load different classes new to their class name to class
file map the longer running request overwrites the cache entry which can
result in constant writes on the class name to class file map cache
file. The "proxy require" file approach mitigates that by creating an
atomic cache entry for every class name to class file mapping.
All this happens at the expense of IO which impact greatly depends on
your opcode cache (you really should use one) and the speed of your
filesystem (if you don't use opcode cache). Basically I'm using the
filesystem as key value store where the filename is the key and the
value the class file path and the aliases.
From the feedback in the community I gathered some more insight into
their configurations and their filesystems. The performance can be
increased by a more performant key value store. My current approach I'm
working on includes getting rid of the proxy requires, the special class
loader cache backend, replacing that one by a more efficient file based
key value store based on a append only btree implementation and beeing
able to use also other existing cache backends as alternative.
Thomas (Tom) Maroschik
TYPO3 CMS Active Contributor
TYPO3 .... inspiring people to share!
Get involved: typo3.org
More information about the TYPO3-team-core