[TYPO3-core] TCA, ext_tables.php and caching concept

[Christian Kuhn]

Hey.

First I'd like to thank Benni for his trust expressed with his latest 
mail in the thread!

I didn't expect such a shitstorm in the '30% slower' thread and it was 
my fault to not give a bigger overview of current work and ideas prior 
to pushing and merging the patch in question.
We raised the topic at the extbase code sprint and realised we must give 
out more details on strategic things the active development group is 
working on, and the vision we share.

When refactoring bigger things, it is often not really clear how the 
final version will look like. We realised the TCA handling must be 
improved to gain speed and to make things easier for developers at the 
same time. With the first patches we got rid of some constructs that 
blocked us from doing things. Two weeks later, we now have a group of 
people aware of the full impact and the system oddities.

Our current concept boils down to:

* loadTCA(), includeTca() are deprecated, not used in core.

* Move all $TCA (ctrl+columns) of a specific table to 
ext:Configuration/TCA/'tablename'.php

* Deprecate ['ctrl']['dynamicConfigFile']

* First transparent cache level: Change bootstrap: On first access 
(caches empty), before loading ext_tables: Find all loaded extensions, 
find files in their Configuration/TCA folders, execute them (they will 
return an array with TCA config, filename is the tablename). This will 
build 'main' TCA. Cache the full result for next operation (cache_core). 
This operation will be done in all scopes (fe, be, ...). ext_tables.php 
*should* not include 'ctrl' sections of tables anymore, but they *can*, 
for compat reasons.

* In FE, caches cleared: Additionally to above, call all ext_tables, get 
result of full TCA and export to cache_core. Will be required on next 
call. This way, we have a *full* TCA in the frontend in all scopes, 
while not executing ext_tables on every call. ext_tables is not executed 
(like before) if this TCA cache entry exists.

* In BE: Use the 'Configuration/TCA' cache above, but always include 
ext_tables.php anyway (like before).


We are confident this scheme will work out pretty well and gives 
backwards compatibility.

* The Configuration/TCA cache entry can be used every time in the 
backend, ext_tables.php will be smaller and takes less time to execute. 
A pre-compiled TCA array takes (nearly) zero time if loaded from disk 
with opcode cache.

* In frontend, full TCA will always be there, loaded in (nearly) zero 
time, just using some MB more memory, no DB call. Memory impact of some 
megabytes is ok.

* Transparent caching, no need for any developer to think about this 
sort of things if TCA files are located at the correct position.

* The ugly 'dynamicConfigFile' construct is gone.


All in all we think that both frontend and backend performance will 
increase with this concept. Next patches to move TCA around will follow 
soon and after that the cache layers will be integrated. The whole thing 
is *must have* for 6.1, otherwise we will do some sort of roll-back.


Regards
Christian