[TYPO3-core] RFC #5045, #5077, #8649: Import/Export requires lot of memory

[Bernhard Kraft]

Steffen Kamper schrieb:
> 
> I did a simular in mysqldumper. The code is old, so it's not best style, 
> but it works writing to gz.
> I simply used
> $fp = gzopen ($df,'ab');
> gzwrite ($fp,$dump['data']);
> 
> so you append the data without any seek.
> Do you write the data not sequentially?

Probably writing of data would be no problem - the problem occurs when 
reading the data - it can be that when reading the file-container a seek 
is required - seek for gzipped files takes very long as this gets 
"emulated" ...

And I didn't want to reinvent the wheel for compressing / decompressing 
- so I used the same approach (handle the file uncompressed - and peform 
de/compression afterwards on the whole container) ....

If speed is important (because of very restrictive 
"max_execution_timeout") I would have another approach.
Either simply disable the compression feature - using the file-format 
drop-down ... this will result in larger files, but if speed, and 
memory, and disk usage are an important feature better change your 
hoster ;-)

For those cases where even disabling the compression isn't enough - I 
would have an solution. But this is not part of this patch - and I would 
like to implement this step-by-step ... instead of creating a complete 
rewrite of the import/export module and then getting angry and annoyed 
because it won't get into the core ... because of "this and that" issues 
... so rather small parts which make sense - instead of one huge piece 
of code which nobody reviews ...




greets,
Bernhard
-- 
Freiheit ist immer Freiheit des Andersdenkenden.
Rosa Luxemburg, 1871-1919
--------------------------------------------------
www.think-open.at