[TYPO3-ect] Extension Rating System and Reviews Status

[Michael Scharkow]

Patrick Rodacker wrote:

> I am still convinced that there is the need of more transparency about
> the extensions and their features mainly for administrators and advanced
> users, so I will keep up the work if there is nobody else who feels
> motivated to coordinate these issues. What I definitely need is some
> feedback and contribution from you guys who are involved in the
> different parts. Maybe I am on a totally wrong track, so please let me
> know what you think ;-)

Hi Patrick,

since I have had some insights into most of the issues you're touching, 
I'll give my comments below. I very much appreciate that you publicly 
present your ideas, as most of these QA issues have been handled (if at 
all) internally until now. But to the point:

First off, I think your approach is too complex. As Elmar pointed out, 
there are severe motivational and cognitive difficulties to face if you 
want to make QA of extensions reliable.

Ratings are simple and fast but very rough and may be completely useless 
in individual cases. That's why we need a large number of them. Reviews 
on the other hand need a lot of effort and motivation, and they may 
still be unreliable in some cases unless you have certified reviewers. 
While with ratings the law of large numbers will make sure that they are 
reliable (remember that each rating/review is per extension-*version*), 
you need either ratings of reviews or certification of some sort for 
detailed reviews. And even this may help against unreliability, but not 
against reviews becoming old/obsolete really fast.

After having witnessed the failure of *all* reviewing efforts I strongly 
believe that detailed reviews (both security and other) are bound to 
fail because of the huge and growing number of extension-versions. If 
you cannot review all extensions, you need not review any because you'll 
most likely choose those that are already popular/well-rated, so 
detailed reviews don't add much information, plus they become obsolete 
very quickly.

On another note, I'd very much prefer not mixing any official typo3.org 
stuff with the T3N magazine. Not because I don't like the magazine, but 
because it excludes all non-German speakers from those reviews. On the 
other hand, I like Jochen Weilands article comparing galleries very 
much, and I'd appreciate it even more if it was posted in English on 
typo3.org, but I don't think there's a need to institutionalize these 
comparisons too much.

Conclusion: Wait for the simple ratings (they have a free text field) 
and see how much they help structuring TER2 already. Don't invest time 
and work in detailed reviews because they'll most likely fail - there 
are always too many extension versions with too few reviewers whose 
motivation will decrease as they realize that last week's 3-page-review 
is obsoleted by a great new version.

Cheers,
Michael