[TYPO3-core] Caches and Locking

[Markus Klein]

Hi Benni,

thanks you for reading them!

The Locker/semaphore stuff is already under review: https://review.typo3.org/28159
Good news is, that the flock method seems sufficient for our needs, though not really fast, but it runs on all platforms, in contrast to the semaphores, which are Unix only.

The locking itself is quite a tough thing. First major problem is that, if you do anything wrong in this area, you will have very, very, very hard times debugging this.
So I highly recommend to stick to well established algorithms here.

Currently we've been discussing this a lot, especially with the most critical thing during bootstrapping of the Core, the ClassLoader.
This class is very special as it has some severe pitfalls hidden, when it comes to locking. For instance, if you try to create an instance of the Locker class inside the ClassLoader, we'll have recursive calls to the ClassLoader, which would then try to instantiate the Locker again causing an endless recursion. So care has to be taken.

Moreover we currently have severe problems with the caches behind the ClassLoader. If the caches are not persistent and/or not consistent, everything fails. Cache backends that may dispose cache entries, like APC, are not appropriate.
Therefore we will implement a second cache level for the ClassLoader. First level will be a fast, size limited cache (like memcache, etc), which may not contain all cached items, and the second level will be a persistent cache which contains all cache entries all the time.

The access to the caches still needs proper locking for concurrency, though. As outlined in the blueprint, we have one more major problem here:
Any implementation of the readers-writers-problem requires at least a shared counter variable for the number of active readers.
Such a shared counter may be stored in a file, but this is rather slow and I would not consider this an appropriate IPC utility for resources with high access frequency like caches.
Therefore we need to use shared memory. The problem here is that no PHP library for such a purpose is compiled in by default. (There are two of them, but only one being Win compatible as well.)

Making the Core bulletproof for concurrency concerns, such a library is required, which imposes yet another (non-default) PHP system requirement for TYPO3 CMS 6.2.

Kind regards
Markus

------------------------------------------------------------
Markus Klein
TYPO3 CMS Active Contributors Team Member

> 
> Hey Markus,
> 
> I read through the blueprints - and I think having an abstraction for the
> locking mechanisms (having semaphore on unix, fallback on windows) seems
> appropriate.
> 
> As this issue is really critical - how can we proceed in order to get this in the
> Core ASAP and have proper time for testing?
> 
> All the best,
> Benni.