[TYPO3-hci] Usability testing in lab?

[Matthew Manderson]

> I got in contact with the usability lab at University of Trier and would
> conduct the tests (for free). The problem is, that one has to rent the
> lab (thinking aloud, eyetracker, cameras for focusgroups,...).

I think that you having access to a usability lab is fantastic. Thanks for
sharing the idea.

My view is this.

1) Lab testing is a costly but ideal way to expose weaknesses that have not
already been identified. It is also an ideal place to test extremely subtle
changes. For example the eye tracking can measure very small and almost sub
conscious differences.

2) Rather like using a costly music studio, we need to plan ahead for the
time carefully.

3) A rather famous usability expert suggests most problems are identified in
the first 3-5 people and anything beyond that is diminishing returns. 80:20
rule.

4) Why don't we start by generating a wiki record of the usability issues
current TYPO3 users and developers have already identified and look for
emerging patterns. bugs.typo3 and the mailing lists already contains some
items and a further mailing rather like the mascot/logo/bugs promotions to
help enlist the typo3 usergroup into reviewing and submitting their own
bugs.

5a) Personally I think that TYPO3 is already excellent but if there is room
to improve it comes from not being user task orientated. Ie the anticipated
workflow is often interrupted with having to make side-steps before
resuming the flow. By identifying the top 500 most frequent tasks for
example I am sure the bugs will emerge and show how side-stepping can be
avoided and even reduce the number of steps needed.

5b) Along the way there needs to be contextural help to get users past the
current workflow problem. A good example of this is v4.0 has lost the
inbuilt Page TSconfig references. Why. The resultant online reference takes
you nowhere near the actual documentation needed.

6) Not entirely TYPO3 core responsibility but core in the minds of users is
the fact that many popular extensions do not come out of the box ready to
go. They require far too much configuration just to get the initial set up.
So TYPO3 core should consider -TYPO3 ready!- approval for extensions that
plug and play. The learning curve is already high so a new developer using
TYPO3 will appreciate the informed choice between two possible extensions.

7) Extension reviews. These are still poor and much more information should
be provided on each extension for users not able to dig deep into the
extension files. Reviews at present are limited to:

"This is a list of all extension versions which have at least roughly been
checked for possible security issues. Although we have taken great care in
reviewing these extensions we cannot guarantee that all of them are really
secure."
 
There is no usability review here? How long did it take to get installed?
What level of knowledge was needed? Did the extension do the job expected
of it? What additional messing about to remove tables and extension CSS was
needed? etc. These sorts of indepth reviews can be undertaken by
journalists leading to greater publicity for TYPO3 and add valuable content
to TYPO3.org

I understand that just talking about the changes takes time and making the
changes takes even more. However, building a well researched roadmap of the
improvements that can be made allows any future development decision to
better 'usability informed'.

My views, hope it helps.
Matthew