[TYPO3-performance] DataHandler high memory consumption

Philipp Gampe philipp.gampe at typo3.org
Tue Nov 4 12:26:38 CET 2014


Hi Lukas,

Nice analysis btw.

As I helped writing this code, I can tell you why we cannot do better here.
:(

The problem here is that you can use conditions for backend TSconfig. That 
means the parsed TSconfig might be different for each page, because:
1) The page has the PageTS field filled
2) The parent page has a condition for this page
3) The userTS has a condition for this page
4) The global TS has a condition for this page
5) Some condition in all of the above based on any global var

You can also see that the TSconfig might be different for different users 
and groups.
At least we have static settings only and nothing like stdWrap ;)

Therefore you cannot really cache it or what would you use as a key?

Lukas Krieger wrote:

> I can think of two patches immediately
> 
> 1. Storing a serialized array on line 1198 and output the unserialized
> array on line 1161 By doing so, the memory usage drops to 18MB which is
> still big, but not as big as before. Just consider, we are still saving
> 280 strings of ~36KB each in the array => ~10MB memory usage and each
> array is the same. Thats still bad!

This helps in your case, but on a normal call, the function is called for 
the same page very often (50-100+ times).
And this is what it is optimized for.
Optimizing it for mass imports means making slower for regular backend 
operations.

> 2. Using the CacheManager which is also used for caching the already
> parsed TSConfig in the TSConfigParser The parsed TSConfig on line 1186 is
> already cached in the CacheManager and it would be easy to add a cache for
> the combined parsed TSConfig+User TSconfig based on the uid of each page.

You cannot cache the UserTS, because it is different for each user and might 
have some condition based on the current page.
And the userTS must be parsed in between the global and the current pageTS.

Thus if you can find a nice and fast algorithm for a cache key, I would be 
more than happy to introduce a cache here.

Best regards
-- 
Philipp Gampe – PGP-Key 0AD96065 – TYPO3 UG Bonn/Köln
Documentation – Active contributor TYPO3 CMS
TYPO3 .... inspiring people to share!



More information about the TYPO3-performance mailing list