[TYPO3-core] FAL: connecting non-hierarchical file database with metadata in TYPO3 db

[Franz Koch]

Hey Francois,

> I agree that this seems a real issue, as it pretty much block developing
> a FAL driver without importing all data. We already have this issue with
> FE users, for which the Core does a join between fe_users and
> fe_sessions, making it impossible to work without fe_users records. We
> should avoid repeating the same design flaw with FAL. It should be
> possible to import metadata, but not mandatory.
>
> If this is fixable (hopefully it is), it should be considered a bug IMO
> and fixed in 6.2.

sorry for not responding earlier - mentioned project is keeping me busy 
(now with a webservice). Thanks for your feedback. From what I have seen 
it's possible to fix these parts but requires some changes ofc. It would 
already help if custom File/Folder classes could be used for a driver 
and if FAL would pass around objects to the driver instead of just the 
identifier. In my case I have to constantly convert the identifier into 
an object again (or grab it from a cache ofc) to be able to process some 
tasks. FAL is object oriented, so why is only the identifier passed 
around throughout FAL?

What's harder to fix is that FAL is designed around a hierarchical 
structure - but not every driver might be hierarchical. I had to 
implement an attribute based storage, thus no strict tree but only 
search criteria with multiple matches for the same files. While I 
created a fake tree based on attribute branches (attribute->available 
values->matching files) I had to inject a custom search/filter wizard 
for editors for better usability. The fake tree is quite a mess though, 
and the filename filters which are passed as callbacks to the driver can 
become an issue with several thousand files you have to grab upfront 
from a remote webservice and parse locally. Would be nicer to get a 
collection of constraints (like the query constraints from Extbase) the 
driver could parse and process or forward to the webservice/remote 
database for preprocessed filtering instead of local post processing.

Another thing that bugged me was that I had to do extra hacks in the 
driver to fool FAL so that it doesn't try to grab processed files 
because in this case an Exception would have been thrown that the 
FileObject could not be created (IIRC). So I had to return FALSE for any 
requests to the driver asking if either the processing folder existed or 
not etc.
So, to better handle this, local file processing should be optional in 
FAL, or at least not be relevant for creating FileObjects (at least from 
what I remember)

If anybody is interested in more details let me know.

-- 
kind regards,
Franz Koch