[TYPO3-mvc] [solved] Memory Usage with 2000 Domain Object

[Tim Schoch | GSTALTIG]

Hi

Oukay, first thing I found the 6 mb: One of the tested filter was spelled wrong so call_user_func( array( $item, 'get' . $filter ) )
threw a Notice every time it got called. I fixed that, but still got the memory fed up with cached objects.
So my conclusion when iterating over a large amount of objects is:

1. Large Query vs multiple Subqueries
There is near to no difference in terms of runtime but the memory impact with one large query is huge, as all the objects get 
loaded on the first iteration and stored in the session and identityMap. With multiple subqueries only the subset of objects are 
stored and will be cleared to 0 after every subquery is done. For me, 100 items per cycle works fine, but you could reduce this 
number even further to decrease the memorpeak. ( see next adn conclusion )

2. Clear Session and IdentityMap
If you don't need the fetched Object Models you can free the memory by calling 

          $this->persistenceManager->getSession()->unregisterReconstitutedObject( $item );
          $this->identityMap->unregisterObject( $item );
          unset( $item );

for every item you've processed.

3. lazy loading is not the problem
I found, that the lazy loading proxy isn't the problem as we thought first. In terms of performance you should leave it on.

> Conclusion
To collect the data of 2500 Items each with 10 relations ( both 1:n and m:n ) it takes around 30 seconds. The memory peak for the 
whole process is 25mb and after it's finished the used memory has increased by 12mb, but thats mainly because I save the 
generated index to other Objects that are not persistet at this point yet.

For me this solution is ok, I hope this helps others to.
Thank you to Lienhart and Christian

Cheers, Tim