[TYPO3-dev] MENUDATA caching problems

[Patrick Broens]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Most of you are aware (and complaining) about the huge amount of
MENUDATA entries in the cache_hash table. Right after the T3BOARD10 I
did some investigation in this topic and posted the text below to my
fellow core team members. I also want to share the outcome of this
investigation with you and let you know that we, as the core team, are
aware of the situation. We will try to solve this asap, but this will
not be easy, as you can read below, and will probably not be solved in 4.4.

I would like to discuss this topic with multiple people, because a
complete rewrite of the tslib_menu class is involved. If you have any
knowledge or ideas on this topic, or would like to be a contributor, let
me know. We will probably discuss this during the next T3DD10.

<snip>
At the previous T3BOARD somebody posted a bug for the bug auction [1]
regarding IFSUB/CURIFSUB states of a parent page which do not change
after hiding a subpage.

The problem of this bug is that a cache_hash entry with the ident
'MENUDATA' cannot be cleared after a change, because it's untraceable
after a change of a page with for instance 'hide' or 'delete' and will
stay in the cache_hash table forever, until the whole cache_hash table
gets cleared.

This sounded familiar to me, because there are a few bug entries about
filling up the cache_hash table with huge amounts of data [2][3][4]. The
last one describes this second problem pretty well.

This filling up is done with mainly 'MENUDATA' as you all probably know.
Some people are reporting cache_hash tables of 5.3 GB!

After some investigation I've found the problem of these huge amounts of
entries. Menudata is generated for 'almost' each link in a menu for each
page it is displayed on. The reason for this is that the MENUDATA
entries are not state aware and have to be generated for each and every
page over and over again.

An example menu:
link 1
link 2
    link 3
        link 4
            link 5

The above menu does not look so big, but it will generate 5 * 5 MENUDATA
entries in the cache_hash table. You can imagine what happens if you
have a huge menu with expAll = 1, where somebody will hide the submenu's
with javascript or CSS and only show the submenu's on hover. A menu with
5 main pages and each of them has 10 subpages will generate at least
3025 (55 * 55) entries in the cache_hash table, if not more, because it
can grow very big depending on your menu configuration and the amount of
pages.

A lot of this data is redundant, because they are the same for most of
the pages. If I click on links 2 till 5, the data of link 1 is the same
every time, but is written to the cache_hash table for each of these clicks.

The problem is the hash is calculated by the rootline of the page the
link is displayed on, not the page the link is generated for, and
secondly, the hash is not state aware. The state awareness is stored in
the content column of the cache_hash table and if we take it out we will
lose the actual caching right now.

The first problem might be solved by using a extra table with the pid's
the MENUDATA is created for (could be more than 1) and the hash. When
hiding a page we can simply delete the cache_hash entry for that
particular id and the entry in this extra table.

The second problem is much bigger. As it looks to me a lot of
class.tslib_menu should be rewritten because the storage of MENUDATA
should be handled differently, which is quite a task. We need to get rid
of this redundant stuff, if it's even possible.

I would like to discuss this with a few of you and get your opinion
about this topic. Has anybody already done some investigation about this
and would like to share their view?
</snip>

Cheers,

Patrick

[1] http://bugs.typo3.org/view.php?id=13665
[2] http://bugs.typo3.org/view.php?id=10944
[3] http://bugs.typo3.org/view.php?id=5360
[4] http://bugs.typo3.org/view.php?id=5425
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkvFhfoACgkQNgvIvIQSN/zlLACfWiJCIg/ICkWmaWv0nEClZ4SM
BAAAnRCIEaXnANAnmY9opsW/ewfcLgJd
=Cdu6
-----END PGP SIGNATURE-----