[TYPO3-performance] DataHandler high memory consumption

[Philipp Gampe]

Hi Lukas Krieger,

Lukas Krieger wrote:

> 6546 public function getTCEMAIN_TSconfig($tscPID) {
> 6547                 if (!isset($this->cachedTSconfig[$tscPID])) {
> 6548                         $this->cachedTSconfig[$tscPID] =
> $this->BE_USER->getTSConfig('TCEMAIN',
> BackendUtility::getPagesTSconfig($tscPID)); 6549                 } 6550   
>              return $this->cachedTSconfig[$tscPID]['properties']; 6551    
>     }
> 
> 
> At first i thought it would be the caching of the TSConfig for each page
> but my cachedTSconfig array is empty. It is just filled with 280 keys and
> for each key a new array: Array ( [2540] => Array ( [value] =>
> [properties] => ) [2539] => Array ( [value] => [properties] => ) [2536]
> =>...

I guess TCEMAIN part of the default TSconfig is empty.

> Of course it increases the memory usage but it is only a few KB for 280
> pages!
> 
> So i commented out the line 6548 (i do not want to fetch my empty
> TSConfig) and the memory usage dropped to 5MB again! Thats the problem!
> 
> It also explains why there is no benefit of chunking the pages into pieces
> and let the DataHandler process only i.e. 10 pages at the same time. It
> calls external functions and the DataHandler itself does not use that much
> memory. (Therefore creating und destroying the DataHandler is no solution
> for the problem - as i tested before)
> 
> I have to leave now but i will dig deeper into the system and have a look
> at the functions on Line 6548 later:
> $this->BE_USER->getTSConfig('TCEMAIN',
> BackendUtility::getPagesTSconfig($tscPID));

BackendUtility::getPagesTSconfig() does cache the TSconfig too. However it 
will collect and parse all TSconfig. The handle the pageTS it also must 
include the userTS, because the later might override some settings.
It might also be different for each page.

Therefore for each page, the default TSconfig is cached in it's parsed state 
(as array), this includes e.g. the quite large RTE configuration.
This is good for normal backend calls, where most calls to this function are 
related to the same page, however gets in your way if you mass CRUD page 
records.

I am not sure what is the best way to proceed here, because this cache is a 
major speedup for the backend rendering (especially page module). Maybe we 
should add a flag to BackendUtility to turn of caching globally for a 
specific call.

Best regards
-- 
Philipp Gampe – PGP-Key 0AD96065 – TYPO3 UG Bonn/Köln
Documentation – Active contributor TYPO3 CMS
TYPO3 .... inspiring people to share!