[TYPO3-hci] Usability Test

[Sebastian Erlhofer]

Hello Kasper, hello to the others.

You are absolutely right with usability-testing in labs.
I guess your suggested "in office" testing via any connection is a cheap 
and proper way doing it. Although there are pros and cons for both of 
course.

As I read you posting I got the idea, that we have not essentially doing 
the transfer of the screen-record during recording, and - to even get it 
more simple and cheaper - we could perform those usability tests without 
an test leader.

We could setup a typo3-testsystem, design some tasks for the users (as 
video or PDF) and the users start some screenrecord-software (as 
mentioned in other postings, like camtasia or sth like that) and try to 
do the task - without anybody watching him/her. If they have a 
microphone installed (that could be a problem in office!) they can talk 
a bit (thinking aloud) what they are doing.

Methodically speaking this is exactly what should be done in the lab: 
Give every user the same briefing (normally test leaders are trained to 
do this with same words, pronounciation etc.) to lay equal foundations.

The user would upload the video and answer a survey (for measuring 
webexperience etc). We analyze this video. We have to have a codebook 
for this, due to the many transcoders - this brings us in the position 
to get comperable analyzes.
If needed you can call/skype the user after analyzing his video for 
asking some questions (methodically speaking: postactional interview).

Thats just another suggestion for another testing-design :)

I guess we have to solve some problems before in any case:
- How do we catch the users in all expert-levels (novice and expert 
user). I guess getting developers is not that problem. I could imagine 
comming to our "users" via some agencies or project leaders/developers. 
They could send a note to their customers... perhaps anybody has some 
better ideas.
- Second problem: Getting the tasks. I guess it would be usefull having 
a survey done before to clear out the major fields.

These are my 20cent for today, now I gonna catch some easy-to-use 
icecream in the sun :)

Greetings!
-sebastian

Kasper Skårhøj schrieb:
> Hi Sebastian and others.
> 
> I agree that usability testing is something we should do at some  
> point. Today I read about all the methods you mention in details so I  
> feel well informed at this stage. However, I was thinking of a  
> technical solution which could be interesting.
> 
> In a usability lab it seems the primary goal is to record what a)  
> happens on screen and b) the dialogue between the test person and  
> test leader. In addition is is nice to see the face of the test  
> person on video as well - and the pros are also careful to make the  
> whole environment friendly and "homesome" to the user.
> 
> Today we have better technology. Let me describe a solution that  
> would surely work:
> 
> I'm the test leader, sitting in Denmark. Mac.
> Ursula is the test person, sitting at company X in germany and uses  
> TYPO3 2-3 times a week. Windows.
> Kasper calls Ursula on Skype which both have installed. If Ursula  
> hasn't, Kasper uses SkypeOut to call a regular phone.
> On Ursulas computer a VNC server is running. Kasper connects to her  
> server with a mac client. Kasper now sees her screen on his mac in a  
> window!
> Kasper starts a screenrecorder application which is also able to  
> capture the macs output audio hence the Skype-interview along with  
> the screen.
> Ursula receives the URL, username and password by mail and the test  
> begins, screen and sound is recorded on Kaspers laptop directly to a  
> mov-file, easily shared.
> 
> Advantage:
> - No expensive lab and equipment
> - No difficulties with travel etc. Very economic on time!
> - Ursula is _exactly_ in her usual environment - a thing they dream  
> to reproduce at the usability lab!
> - persons from all over the world is tested in a day!
> - It costs next to nothing (important for Open Source Software project)
> - Data is collected in an easily shared format (mov files, mpg,  
> whatever) with not post-processing required.
> 
> Disadvantage:
> - Getting Ursulas screen picture transmitted to Kasper! Most likely  
> she cannot be expected to set up a VNC server, neither will she have  
> access to modify the router/firewall to allow Kaspers remote  
> connection. So the VNC solution is more or less out of the question.  
> BUT...: does another technology exist - such as a Java applet  
> available from a website - which allows to share a desktop for a  
> single session and doesn't give problems with firewalls etc?
> 
> 
> 
> - kasper
> 
> 
> _______________________________________________
> TYPO3-team-hci mailing list
> TYPO3-team-hci at lists.netfielders.de
> http://lists.netfielders.de/cgi-bin/mailman/listinfo/typo3-team-hci
> 


-- 
Sebastian Erlhofer
medienagentur mindshape freiburg / trier
Jakobstraße 31
54290 Trier

mail erlhofer at mindshape.de
http www.mindshape.de
fon  0651.4639822
fax  0721.151368557