[TYPO3-mvc] scheduler problems - memory exhausted

Tim Schoch | GSTALTIG tim.schoch at gstaltig.ch
Mon Mar 26 12:00:18 CEST 2012


Hello Stig
I had similar problems when trying to handle large customer databases in tasks.

The problem was, that when calling clientRepository->findBy*() and then looping them in a
foreach() all clients got substituted at once and therefor using memory.

I found three solutions: 
One acts like the pagination widget: before looping the records i alter the limit to only get the first 50 records.
The next time the cronjob is called I get the next 50 clients. This works pretty good for mailings and acts similar to direct_mail.

The another solution would be to use raw queries that return an array instead of the object. Usefull if you eg only need the email adresses of 10'000 clients...

The last (and least recommendable) solution is to, again, get only a subset of objects at a time and process them. But then instead of ending
the task manually unset the objects and go start over again getting the next bunch of records.
(IIRC all objects are cached inside the objectManager, but could be somewhere else)
This method, ofcourse, is far from using api methods and could break once extbase gets an update... but it works if you need to process
huge amounts of data in one run and need the extbase objects. (eg: use solution 2 if you dont need the php functionality provided by your domain models)

Hope this might help you finding a good solution for your task
Tim

GSTALTIG GMBH / www.gstaltig.ch
Sonnenhofstr. 13 / 6020 Emmenbrücke
tim at gstaltig.ch / 076 488 24 97



More information about the TYPO3-project-typo3v4mvc mailing list