[TYPO3-core] RFC: Blueprint database service

Jigal van Hemert jigal.van.hemert at typo3.org
Sat Jun 21 23:16:46 CEST 2014

Hi Thomas,

I've waited a few weeks to provide enough time for people to read the 
blueprint and think about it. Thank you for your extensive answer!

I'll have some questions and remarks between your thoughts:

On 4-6-2014 17:24, Thomas Maroschik wrote:
> Thanks for that blueprint. I really like concept and the approach of
> integrating Doctrine DBAL. Further I don't see the need to integrate ORM
> now, but we should not obstruct the possibility.
> I propose a slightly different architecture that mitigates some of
> issues I discoverd in TYPO3 DBAL through the last years.
> ----------------------------------------------------------------------
> class ConnectionPool (singleton)
>    * contains all connections
>    * not only a single global connection,

The problem of handling connections is in my concept moved to a lower 
level. When the SQL query needs to be constructed it is only possible to 
determine to which DBMS to connect based on the table usage.

There is a use case for requesting a new connection: nested queries.

>      because different connections may
>      * have different transactional behavior

Well, the situation is worse: in MySQL certain table engines support 
transactions, others don't. Maybe we can provide a method to start a 
transaction with the next query which may produce some error if the 
DMBS/engine doesn't support it.
If we switch connection before the transaction is ended the transaction 
needs to be rolled back.

>      * conflict with lastInsertId
>        (eg hooks inserting stuff directly after the DB insert)

Perhaps we should store this by default with every insert?

>    * connections have an identifier
>      * _DEFAULT is the identifier for default connection
>        (similar to DBAL handlerKey)
>    * a connection will be demanded with the class name of the class
>      the connection is used in as identifier
>      * eg $connectionPool->getConnectionForClass('DataHandler')
>      * this way it will be possible to configure different connections
>        for different classes
>      * mocking of database connections in tests is easy then, can be
>        registered temporarily in connection pool
>      * if no specific connection is configured,
>        default connection is taken
>      * it will be easy to integrate DI this way in the future

Can't we hide the whole connection handling from the caller? The calling 
code isn't aware of switching connection if some of the tables are in 
different databases.
Can you explain why the calling

> class {Plattform}QueryBuilder extends Doctrine\DBAL\Query\QueryBuilder

The Doctrine query builder and expression builder are unfortunately not 
cross DBMS. They warn that you have to make sure that the db supports 
all the features you use.
In order to support functions like find_in_set() like our current DBAL 
does for other DBMS, we need to have DBMS specific implementations.

> Schema Comments
>    * schema should be represented in form of migrations (up/down)
>      that use the schema manager of DBAL
>      * [2]
>    * in schema plattform portable types provided by Doctrine DBAL
>      should be used

Looks sensible, but we'll have to make sure that the portable types are 
translated to the correct column types. Also, we need some mechanism to 
support ext_tables.sql files and convert them into into schemas/migrations.

> Goals of the architecture
>    * in order to be able to use more transactions in TYPO3 we need
>      more connections simultaneously, thus a connection pool

Transactions are usually a good thing; they are however not always 
supported. Current DBAL supports XML as a storage. MyISAM table in MySQL 
have no support for transactions and if we support NoSQL databases in 
the future there is also a problem with transactions in many cases.
Plus, if the queries in a transaction use different connections a 
transaction is not possible. The calling code cannot predict this 
because some of the core tables may be stored in a different database.

>    * singletons should be rarely and only used when really necessary
>    * sane defaults, flexible configuration
>    * assignment of individual connections to classes
>      * use case: only a single class uses that other database and
>        doesn't need the virtual connection

How can the class know where the information is stored? This is done on 
a per table basis.

>    * the layer on top Doctrine DBAL should be as light as possible
>    * make our layer also usable for Doctrine ORM for the future

Thanks again for taking the time to analyze this!

Jigal van Hemert
TYPO3 CMS Active Contributor

TYPO3 .... inspiring people to share!
Get involved: typo3.org

More information about the TYPO3-team-core mailing list