[TYPO3-mvc] findOneBy memory usage

[Saverio Vigni]

I've digged deeper in the extbase code and i think the problem is in the 
Tx_Extbase_Persistence_Mapper_DataMapper::mapSingleRow, after a new 
object has been created it's being stored internally in an array:

$this->identityMap->registerObject($object, $row['uid']);

registerObject stores the object twice:

$this->objectMap[$object] = $uuid;
$this->uuidMap[get_class($object)][$uuid] = $object;

so everytime i call the getFirst method the new object is stored in 
memory, in my case i have a very low reuse of objects since it's rare 
that i'll found the same page linked twice or more in every iteration.
Immagine i spider 50 pages for each run of the scheduler and each page 
has 50 links, for each link i have to check using the findOneByUrl if 
the page is already in the db, this creates in the property mapper 5000 
objects in the worst case, just to spider 50 pages...

i was wondering, is all this overhead really necessary? why?

Thank you
Saverio

Il 18/07/12 09.18, Sebastian Fischer ha scritto:
> Hi Saverio,
>
> i cant explain why it happens but i can explain the when.
>
> You described that it happens after you hit the result with the 
> getFirst method. Thats quite correct because until then the real 
> result was not taken from the db. So until you use getFirst no real 
> model and nothing from the persistence is taken into account beside 
> that a query_result object was created.
>
> Hope that explains why all is fine until you get the item.
>
> Greetings
> Sebastian
>
> Am 18.07.2012 02:28, schrieb Saverio Vigni:
>> Hello everyone i have a problem with the findOneBy
>>
>> I'm making a simple web spider for checking the link structure of a
>> quite big vacation rental website, everything goes fine until I check in
>> the link table if the link I found in the page already exists, actually
>> I'm using the code:
>>
>> $newPage = $this->pageRepository->findOneByUrl($link->getUrl());
>> if(!$newPage){
>>      $newPage =
>> $this->objectManager->create('Tx_Webspider_Domain_Model_Page');
>>      $newPage->setUrl($link->getUrl());
>>      $this->pageRepository->add($newPage);
>> }
>>
>> this is inside the loop of the links found in the page I'm processing,
>> the cal to the findOneBy makes the memory usage grow from 48MB to
>> hundreds of megabytes.
>>
>> I tried rewriting the function and the problem happens when I try to
>> access the result of the query with the getFirst() method.
>> I haven't been able to understand if the leak is in my code or in
>> extbase, can anybody help me?
>>
>> Thank you
>> Saverio Vigni
>> http://www.hor-net.com
>
>
> _______________________________________________
> TYPO3-project-typo3v4mvc mailing list
> TYPO3-project-typo3v4mvc at lists.typo3.org
> http://lists.typo3.org/cgi-bin/mailman/listinfo/typo3-project-typo3v4mvc
>