[TYPO3-dev] linkvalidator's memory footprint

[Christian Lerrahn]

Hi Philipp,
On Fri, 22 Jul 2011 13:21:53 +0200
Philipp Gampe <typo3.lists at philippgampe.info> wrote:

> Christian Lerrahn wrote:
> 
> > Why does linkvalidator use up that much RAM? Does it possibly hold
> > stuff in memory where there is no need to?
> 
> Might be, the code is not optimal, but I think the real problem is
> that you are checking a link that links to a large file.

That's not impossible of course. However, how large is "large" that I
run out with 1.5GB available. I'm sure that I'm not linking to any file
more than half that size if at all.

> Do to some problems with web servers not supporting HTTP/HEAD (e.g. 
> amazon.com) we have to use GET on the links. But this loads the whole
> link into memory. So if you link to a large file, you are in
> trouble :/
> 
> If this is the case, you might want to try df_tools for externals
> links, as this checks each link individually and I think you can
> blacklist links there.
> You can use linkvalidator for all other links then.

I had a quick look at df_tools but found the manual to not be the
greatest. So, this will probably have to wait until I have a bit more
time... :(

Cheers,
Christian