[TYPO3-dev] Improve EXT:coreapi [Google Summer of Code project]

[Helmut Hummel]

Hi Stefano,

On 05.05.14 23:09, Stefano Kowalke wrote:

> I am one of the accepted students for GSoC this year. Felix Oertel
> (@foertel) and I doing a mentor-student tandem because we both live in
> the beautiful city of Flensburg.
>
> I applied for improving the extension CoreAPI[1] which was originally
> initiated by Tobias Liebig (@etobi).

Great to hear! I'm looking foreard to the results!

> In my application I add these goals
> which I like to accomplish:
>
> * manage users (list, create, update, delete)
> * lock/unlock the TYPO3 backend
> * run/check the reports from the reports module
> * list, get and set TYPO3 configurations
> * create a database dump
> * print the page tree and partial trees (on CLI)
> * list content/records on pages
> * generic create/update/delete records (and not doing the plain SQL, but
> using the DataHandler (aka tcemain)!)
> * update an extension from TER

I would like to suggest a meaningful grouping:

== Manage Data (aka. CRUD actions of TCA data) ==

This would cover "manage users", "list content/records on pages", 
"generic create/update/delete records with DataHandler"

This is what I did years ago[1]. The code is a bit outdated and in some 
areas coupled to a XMLRPC implementation but the entry API is quite clean[2]

It may serve as some inspiration.

There are a lot of things to consider here to have a nice and useful API 
in the end which does a bit more than just CRUD (hide records, move 
records, translate records, copy records (recursively) etc.) Just look 
at the DataHandler data structure for processing 
($tce->process_cmdmap(), $tce->process_datamap()) for additional hints 
what actions could be useful in this API.

"print the page tree and partial trees"
This is something I wrapped my head around a lot.
I do not have a good concept, but reading all the above and dealing with 
records that are nested in trees, it might be a good idea to look at 
PHPCR[3] and/or TYPO3CR[4]. If not using the code, I bet it would be 
benefitail to get inspired by these concepts, as although in TYPO3 CMS 
everything is a database table, there is always a notion of hierarchical 
structure: every record lives on a page (node) thus can be considered as 
child node...
Printing a tree would then just be a matter of serializing a node structure.

Ah, and another thing here: What about assets (files)? I think PHPCR has 
some implementation and Flow has a ResourceManager. Not sure if this 
fits here, but (repeating) maybe worth considering.

A benefit from using DataHandler would be to be able to perform actions 
in a workspace, without changing anything.

This leads to something which needs to be considered when implementing 
such API:

== Authentication ==

DataHandler needs a valid (backend) user to perform actions.
since coreapi is a command line tool which (normally) is assigned to a 
single user there would be a need to for a concept how to handle that.

It would be good to at least consider that when creating a API for 
DataHandler actions, how we handle different 
users/roles/permissions/workspaces?

How are accounts authenticated and how does the API interact with this 
layer?

When doing a "Manage Data" API this has to be taken into consideration.

== Configuration (aka TYPO3_CONF_VARS) Handling ==

"list, get and set TYPO3 configurations"

This would be great already. I would suggest here, that there are 
different kinds of configuration (TYPO3_CONF_VARS, TCA, TsConfig, 
TypoScript, Extension Configuration ...)

When designing the API for that, it would be great if you could at least 
consider the case that there are different configuration sources which 
need different configuration writers (not saying everything needs to bi 
implemented, but at least considering if it would make sense to have a 
unified configuration API, which can handle different (maybe not all) 
configurations

== Reporting ==

"run/check the reports from the reports module"

A good start! There are other places with reports: Install tool, log 
files, maybe more. It would be nice to evaluate if it makes sense to 
have a unified API for all, or if they differ too much (which most 
likely is the case).

== Extension Handling ==

"update an extension from TER"

This could be one thing in a group of others, like uploading to TER, 
(de-)activating, removing, configuring (would this fit to configuration 
handling?), listing, details of extensions.

== Maintenance Tasks ==

"lock/unlock the TYPO3 backend" is already implemented in an old school 
cli_dispatcher command (and in my typo3_console project where I put the 
functionality into a command controller)

There are different other (easy) maintenance tasks that could be wrapped 
in an API, like putting FE in maintenance mode, create login messages 
(these are regular records, but it still would better fit in this API)

> Now I am asking you (as the community of TYPO3) which features do you
> like to see in the CoreAPI. It would be nice if you could vote for one
> or the other feature, so that we can compile some milestones out of this.
>
> Thanks in advance
> Stefano
>
> [1] https://github.com/etobi/ext-coreapi
> [2] http://wiki.typo3.org/Gsoc2014/Ideas#Extend_EXT:coreapi

== Import/ Export Data ==

"create a database dump", uhm, well not really.
mysqldump does a good job here ;)

But we have an import/export feature, which works pretty nicely.
What about having an API for that (easily defining what to export and 
where to import)



= Conclusion =

Considering all this (not to speak from implementing) would take months, 
and I think you should focus on a few items only. But it would be nice 
if the result could have a notion of the "big picture" I tried to outline.

If we do this right, a REST API will surely be on the horizon ;)

Have fun and feel free to contact me. I'm happy to discuss these things!

Kind regards,
Helmut

[1]http://forge.typo3.org/projects/extension-typo3_webservices/repository/entry/typo3_webservice/trunk/library/api/

[2]http://forge.typo3.org/projects/extension-typo3_webservices/repository/entry/typo3_webservice/trunk/library/api/Typo3Rpc_Records.php

[3]http://phpcr.github.io/

[4]http://forge.typo3.org/projects/package-typo3-neoscr

-- 
Helmut Hummel
Release Manager TYPO3 6.0
TYPO3 Core Developer, TYPO3 Security Team Member

TYPO3 .... inspiring people to share!
Get involved: typo3.org