[TYPO3-Performance] General questions

[Dmitry Dulepov [typo3]]

Hi!

Mathias Schreiber [TYPO3] wrote:
> Luckily we already did (almost) a month of full time work and dug deep 
> into the "too many queries" problem.
> 
> That's why I wanted to start there.
> 
> What you can do is set up a site with a lot of content and a rather big 
> pagetree.
> 
> Then xclass t3lib_db so every query gets stored in a seperate database 
> table.
> 
> After you surf around the site for some time do something like this:
> SELECT COUNT(*) as brace_yourself,query FROM your_log_table GROUP BY query.
> 
> You will see that a lot of queries run multiple times.
> E.g. Let's say you hit one page 20 times - this should result in a 
> queryCount of 20 for query A (at least this would be the ideal situation).
> But query A is executed over a thousand times.
> So something does go wrong (please correct me, if i'm wrong).

I'd say that repeating queries are bad. They appeared due to a long history of TYPO3. As you know TYPO3 development was a bit chaotic (no planning) till recent times.

In my opinion we should take a profiler and see what takes most time. Then we should optimize those areas. Removing queries will pop up in this process for sure. But we should get a picture what is slowest first.


>>> I think it is dangerous to claim that uncached content isn't a 
>>> problem if it takes ages to render, because everything is good once 
>>> it is cached.
>>
>> No one claimed it.
> 
> You didn't, on the snowboard tour I heard different stuff about this 
> issue (which gives me BAAAAAAAAD headaches) :)

Oh, I misunderstood you, sorry :( You are completely right. If non-cached page generation takes ages, it is bad!

>> You know why? ;) For example, get rid of some workspace queries in the 
>> core and you will see a performance increase.
> 
> Yes please :)
> We get along without workspaces very well.
> But I know that unfourtunately making the use of WS inside TYPO3 
> configurable sounds like a pretty tough task.
> A nice thing would be some sort of "I don't use WS - skip this part of 
> the processing" switch.

I think it is another planning problem in TYPO3... :( I think workspaces were created as experiment with excitement. There were no really planning for them. I think it was like patching one place, then another place, then yet another... As a result, workspaces are spread over core without any way of removing them. Checks and queries are everywhere and they run always, even if web site do not use them. Changing it would be difficult.

> But you will also understand that I am frustrated when I see great 
> concepts not being done properly.

My feelings as well!!!

> Like the rootline caching.
> Cache = good
> Nuking cache each loop = ...I can't think of a nice way to describe this 
> - it's... I don't know, but you get the point.

There's more to this... For example, I have comments extension. When comment is added to the news item, I must to show this comment. There are two ways:
- make comments USER_INT (non-cacable), which is bad by itself because in most cases comments are read, not created.
- clear cache for the news item. That's perfect solution except... Except that it is impossible :( I can only clear cache for the whole page. So if I got 10000 items in cache, I have to clear them all once I change one item.

That's again an example of pure design and planning. I tried to change that and add better way of clearing cache but that would take hours of work and lots of changes in the core. I cannot afford it because I got only 4h/week now for the core :( And every extension has the same problem if it does something for single view in tt_news-like extension.

> Anyways:
> a good thing would be if we ALL would put together all our experiences 
> in one place.
> Then define a way how patches are being applied to the core branch.
> 
> I fear that (as so often) we work hard and the desperately needed 
> patches are lost in someones mailbox (happened to me 4 times, so attempt 
> 5 has to work, otherwise I gotta do my own thing).

That's why we have bug tracker ;)

> So I'd like to have one "official" coredev in charge of the results of 
> all this discussion.

We do not have official developer for performance, neither we have a performance team. I made some performance-related changes and I think Ingo plans something.

I am good at optimizing queries, so I can do that for TYPO3.

-- 
Dmitry Dulepov
TYPO3 Core team
My TYPO3 book: http://www.packtpub.com/typo3-extension-development/book
In the blog: http://typo3bloke.net/post-details/typo3_code_formatting_in_eclipse/