[TYPO3-english] Very slow performance when clearing cache & viewing template / huge db size increase

[Dan Osipov]

I wrote a blog about hacking tt_news to optimize it for performance:
http://danosipov.com/blog/?p=21

This helped significantly, but is still a problem when cache clears on 
our site during peak traffic times.

Dan Osipov
Calkins Media
http://danosipov.com/blog/

Francisco Javier Huerta Lopez wrote:
> Hi, thanks for all your suggestions:
> 
> 
>> Hi Javier,
> 
> 
> Ø  please turn on Mysql logging for slow queries. Then check the indexing
> 
> Ø  of tables causing the slow quieries. In most cases adding additional
> 
> Ø  Index might help to speed up.
> 
> 
> Ø  Do you already use any accelerator for PHP?
> 
> 
> Peter, we'll be turning on the slow query log today. We know it's a db issue by now, we are just wondering what could cause it to grow up so fast, and become so slow. I'm not so sure a PHP accelerator may be of any help, but, what would your suggestion be? So far, the site is very, very fast for users (we have an HTTP cache / accelerator array).
> 
> 
> 
> 
> 
> Ø  Apparently your cache tables have suddenly started growing much more
> 
> Ø  than usual. I'm experiencing similar symptoms on www.tue.nl where we
> 
> Ø  have around 20k published pages. The part of the DB containing the
> 
> Ø  actual content is little over 600MB at most, but the cache quickly makes
> 
> Ø  that grow to well over 4GB within a day.
> 
> 
> 
> Bas, your site seems very similar to ours in size. And yes, we are going through the same problems you are experiencing. At first, we thought it was an indexing issue, but it's quite obvious by now that the problem is the cache implementation. It was by far worse when we had a "Latest pages" plugin - we learned the hard way, and took out all the plug-ins we really didn't need. How did you build your cronjob? Are you going into mySQL and cleaning the cache tables manually, or can Typo3 be configured to clean them? Our back-ups have, obviously, skyrocketed in size lately.
> 
> 
> 
> We've been using Typo3 for close to 8 years now, and can't help but be amazed at how flexible and powerful it is when compared to commercial offerings. But this performance issue could be a deal-breaker in a couple of months, since we are planning to offer customized content. I'd assume this would place so much extra stress as to require extra hardware on our side. It's good to know a file cache project is ongoing - maybe this is the answer to our problems.
> 
> 
> 
> Ø  Do you have any calendar display extensions? Sometimes extensions or scripts
> 
> Ø  can effectively create a very large number of pages in the cache when
> 
> Ø  spidered by search engines (eg daily views have 'view next day' and go on
> 
> Ø  for years). Similar things can happen when you have multiple routes into
> 
> Ø  the same content that have different getvars and chaches.
> 
> 
> 
> We don't have that many extensions - the main one we use is tt_news, actually. We have *lots* of news. Maybe this could be the problem?
> 
> 
> 
> Thanks for all for your help.
> 
> 
> 
> Javier.
>