[TYPO3-ect] TER clean up actions

[Jigal van Hemert]

TER Cleanup

The text below will be put in the ECT wiki too. It outlines the steps we 
need to take, technical considerations, time frame, etc.

Steps
1. Communication
2. Modify TER_fe search
3. Modify EM
4. Add Reports item
5. Periodically mark extensions in TER as outdated
6. Support in third party extensions

1 Communication

1.1 Announcements

  * news, buzz
  * mailing lists

The TER cleanup will be announced in as many places as possible. The 
announcement will explain:

  * which extensions are affected
  * the impact of existing installations (after core update)
  * actions which extension authors need to take
  * actions which site owners can take
  * consequences for security issues

1.2 Mailing extension authors

  * extension key owners
  * authors in ext_emconf.php

Extension authors will be notified that their extensions will be marked 
as outdated after a grace period (6 months?). The notification will also 
tell what actions an extension author can take to prevent the extension 
from being marked as 'outdated'.

2 TER fe search

2.1 Exclude outdated extensions from standard search

Extensions which are marked as 'outdated' will be excluded from the 
results of TER search. As the new ter_fe plugin will most likely be used 
on the typo3.org website when the grace period has ended, these changes 
need to be included in the new plugin.

2.2 Extra search for outdated extensions

An extra search for 'outdated' extensions needs to be created or a 
search option needs to be added to include 'outdated' extensions in the 
search results. This will also be included in the new ter_fe extension.

3 Extension Manager

3.1 Status for outdated, installed extensions

Extensions which are marked as 'outdated' need to be marked accordingly 
in the Extension Manager. The UI design team will be asked to make a 
visual design for this. Patches for all supported branches need to be made.

3.2 Force TYPO3 dependency setting on upload

New versions of extensions need to have a dependency on TYPO3 for at 
least the lowest currently supported branch. When uploading an extension 
this dependency must be checked and missing or too low dependencies must 
be rejected. Existing EM version already send dependencies to TER and 
can handle error messages. This is a change in the TER server, but it 
requires knowledge in the TER server of the currently supported versions.

3.3 Hide outdated extensions in TER search

For this change there are two different situations:

  * existing installations which are not updated (yet)
  * new or updated installations

To hide outdated extensions in existing installations we could use the 
field reviewstate. They would then become 'removed as insecure'. See 
section on technical implications.
Installations of supported branches will have an updated EM which hides 
outdated extensions in search results.

4 Reports module

4.1 Reports module: List outdated extensions

Depending on the technical implementation the Reports module will also 
show which installed extensions are marked as 'outdated'. At the same 
time we can list extensions which have an upper limit in the TYPO3 
dependencies which is lower than the current version (this can happen 
after a core upgrade). This will help integrators to take appropriate 
actions during a test upgrade.

5 Periodically mark extensions in TER as outdated

5.1 Periodic task: extensions with max dependency < deprecated

Whenever a branch is declared EOL extensions with a upper limit in the 
dependency settings of this branch will be marked as 'outdated'. This 
can be implemented as a script which checks the entries in the TER 
database. Running this script must be added to the procedures for the 
Release Managers.

5.2 Periodic task: extensions without dependency which are older than x 
years

Because a lot of extensions do not have dependency settings yet the 
script of the first cleanup can be used periodically to mark old 
extensions as 'outdated'.

6 Support in third party extensions

Once the changes in TER/EM are known and the technical implementation is 
clear authors of known third party extensions which provide monitoring 
services will be notified of these technical changes.

Extensions which provide such a service are:

  1  caretaker
  2  nagios
  3  ...

7 Technical considerations

With the cleanup action and the changes we have to try to maintain as 
much backwards compatibility as possible. Solutions which hide 
'outdated' extensions in unmodified Extension Managers are preferred.

7.1 Using reviewstate setting

Currently the EM checks for reviewstate=-1 to detect insecure 
extensions. The Security Team already mentioned that 'outdated' 
extensions will not have their focus. In this light we could consider 
them as being insecure because they are outdated, unsupported and not 
maintained. Using these extensions will be the sole responsibility of 
the integrator.

7.2 Difference between 'insecure' and 'outdated'

For updated EM versions there should be a difference between 'outdated' 
and 'insecure' extensions. In the EM only extensions with a review state 
greater than or equal to zero are shown. Extensions with a review state 
of less than zero are marked as 'insecure'.
In this light the proposal is to use a review state of -2 (minus two) to 
indicate 'outdated' extensions.

7.3 Dependency check in TER server

The TER server needs to be aware of 'current' versions. There are 
already scripts such as one by Xavier Perseguers [1] which provide 
version information. A similar script (extended with oldest_supported) 
could be used by the TER server to check dependencies.
The Extension Manager already supports displaying error messages from 
the TER server, so this only requires changes in the TER server.

8 Next steps

1) check with Security Team, server team, new typo3.org team, Release 
Managers, UI/design team
2) enter tasks in forge and prioritize them

9 Time schedule

  * Announcements: a.s.a.p.
  * Mailing extension authors: a.s.a.p. after getting addresses, etc.
  * Marking extensions 'outdated': August 1st 2012; which means a grace 
period of five and a half months to add dependencies and upload a new 
version
  * Changes in extensions on TER server and EM: published before July 
1st (which means they must be included in core releases before this date)

[1] http://typo3.causal.ch/releases.php

-- 
Kind regards / met vriendelijke groet,

Jigal van Hemert.