[Typo3-typo3org] some thoughts on TDR (or documentation.typo3.org)

[Michael Scharkow]

Hi all (and sorry for x-posting, I thought the doc people might be 
interested as well),

I have done a little research on the future doc repository and want to 
share some ideas. This is all rather loose ends but if there's interest 
and I'm not stepping on anyone's toes I could write a proposal as Robert 
has done for TER 2, only less elaborate probably...)

1. Basic file format

This is and will be OOWriter. We need to support sxw and the new odt 
format which seems to be the future. There have been discussions on 
Docbook and other formats but for a number of reasons[1] we're stuck 
with OOWriter.

Now to the workflow...

2. Document input
OOWriter-Files are uploaded
a) directly to docs.t3.org
b) by extracting the file from /doc directory during extension upload 
(legacy feature)
c) via some import structures for wiki-pages and timtaw. For this, we 
might offer a feature that converts a snapshot of the page content to an 
  OOWriter file (Robert says this is possible) that is then processed 
just as any other doc.

Additionally, extension authors are able to download a customized 
template based on metadata of their ext and maybe a few additional form 
inputs.

3. Document processing
When a doc is uploaded, meta-data is updated in the db (and/or the 
file?) Ideas for meta-data: author(s), title, version, categories, 
translations. dependencies (= required reading), related reading, ???

Then the file is stored in the filesystem, just like in TER with some 
hierarchy /docs/extensions/t/e/templavoila/manual.sxw
Another question is the path and directories, i.e. would it be enough to 
have /docs/extensions/t/e/templavoila.sxw? Simple versioning could be 
implemented in a way that we use a symlink manual.sxw to manual_x.y.z so 
that the current version is always used but older versions are 
available... But this does not scale too well.

Since everything is on a plain filesystem, replication with rsync and 
stuff should be a breeze.

When a doc file has successfully been uploaded, the cache for the HTML 
output is cleared, the translation team is informed, and file conversion 
is started/scheduled.

NOTE: Let us pleasepleaseplease get rid of the section mapping stuff 
that exist in the current TER. I don't understand why we need it when we 
can simply extract all the sectioning information from the source file.

4. Document output
Output on docs.t3.org is done with Robert's office displayer which seems 
to work well and offers everything we need. Additionally, we can render 
PDF's either on upload or with a cronjob for every OOWriter file.

Concerning the last point: I have successfully managed to batch-convert 
SXW to PDF through the native Openoffice Export. With some help from an 
online-converter script and a little python, this works pretty well. 
There are a some caveats though: 1. You need Openoffice on the server, 
2. run it without X (probably) and 3. it's not too fast of course.

On the other hand, I see this as the way to go because
a) it's the most robust solution because the OO-PDF-Export should work 
with all exotic formating and layout stuff. The PDF's look like the SXWs 
which is important for the GUI-oriented people here.
b) it's future proof because the export will work with the new 
odt-format as well and we can use it as soon as OO 2.0 is out.
c) all other solutions sucked in one way or another

Thanks for the patience ;)

Greetings,
Michael



[1] These are mainly: Free Software, GUI available, legacy reasons, 
office_library works already in T3.