[TYPO3-mvc] Memory Usage with 2000 Domain Objects

[Tim Schoch | GSTALTIG]

Some after thoughts

I loop each item and use this function to test if the relation was set (remember, its optionally )
---------------------
    // predefined filter array
  $filters = array('Color','Difficulty','Flavor', ... );

...

    // loop items inside subquery
  foreach( $subQueryResult as $item ) {
      // save category uid for later use 
     $currentCategoryUid = $item->getCategory()->getUid();
      // loop through filter
    foreach( $filters as $filter ) {
        // assign the filter if the getter returns a value
      if ( call_user_func( array( $item, 'get' . $filter ) ) ) {
        $index[$currentCategoryUid][$filter] = $filter;
      }
    }
  }
---------------------

As the Filter are set to lazyloading, it would be nice to check if they're set without having to load them.
Is there a 'default' or a better way for this?

Cheers und "än Guete"
Tim

ps: sorry to the Webreaders, I forgot to manually break the lines...
  _____  

From: Tim Schoch | GSTALTIG [mailto:tim.schoch at gstaltig.ch]
To: typo3-project-typo3v4mvc at lists.typo3.org
Sent: Tue, 22 Mar 2011 12:22:18 +0100
Subject: [TYPO3-mvc] Memory Usage with 2000 Domain Objects

 
        
  Hello List
  
  I'm trying to aggregate an index with something around 2000 items and all I get is a «Fatal error:  Allowed memory size of 171966464 bytes exhausted». =[
  
  For each item I have up to 7 optional 1:n Relations (lazy, Value Objects) and 2 n:n Relations(lazy, Value Objects). If the item has a filter, I add the filterUid/categoryUid combination to the index. The fully populated index has around 300 combinations, stored in an array like this: $index[categoryUid][filterUid] = filterName. The size of the index itself can't be the problem. So I've tried to track down where the memory is cluttered and found some ways to cut down the memory footprint, but it's still not good enough:
  
  
  1. Subquery ( copied the code from the paginate widget )
  $all = $this->repo->findAll();
  for( $page < $totalPages ) {
    $subQuery = $all->getQuery();
    $subResults = $subQuery->setLimit( 100 )->setOffset( 100 * $page )->execute();
    foreach( $subResults as $item ) { ... }
  }
   (code is shortened)
  
  
  2. unregistering Items
  $this->persistenceManager->getSession()->unregisterReconstitutedObject( $item );
  $this->identityMap->unregisterObject( $item );
  As I don't need the items for further operations I added this call inside the foreach( $item ) loop. This removes the items from the session and identity map cache but it hasn't that much of an impact (around 1.5 kb per item or 3.5mb for all items).
  
  
  3. unsetting what's not needed
  unset( $item );
  unset( $subResults );
  unset( $subQuery );
  I tried to unset everything that I don't need, but it's not that much of an impact, as it gets overridden by the next foreach-loop anyway. ("saves" 20kb per loop)
  
  ---------------------------
  
  I added some memory_get_usage() calls to measure the impact and found out, that every Subquery leaves me with a memoryleak of aprox 6 mb. When debugging, i can't find where these 6mb are comming from?
  
  Item No     /   Begin Loop    /  After clean up  /
  000 - 099   /    34717432    /        41533200  / + 6815768
  100 - 199   /    41534136    /        47588240  / + 6054104  < the index is 99% completed after the second run
  200 - 299   /    47588936    /        53908520  / + 6319584
  300 - 399   /    53909016    /        59850248  / + 5941232
  
  ---------------------------
  
  After each sub query run, the sessionManager shows exactly 9 Objects, so the unregistering is successfull. I've looked into every single dispatcher object using PhpEds debug tool and found no other occurences of any of my objects... but still... after 20 sub query runs, the 6 mb add up to 120mb...
  
  Are there further ways to decrease memory usage of extbase, when dealing with complex objects?
  
  Thanks Tim
  
  
      
  _______________________________________________
  TYPO3-project-typo3v4mvc mailing list
  TYPO3-project-typo3v4mvc at lists.typo3.org
  http://lists.typo3.org/cgi-bin/mailman/listinfo/typo3-project-typo3v4mvc