Usability Testing

One of the big things that separates UX from UI is user data. A UI can look stunning, but if it operates badly no manner of awesome graphics will make it work. UX requires data not opinion to function properly. Identifying not only what people say they want but what they don't is critical; what a user needs and how they behave and actually operate a systems is the core of UX. There are scores of ways to get this data but far and away the best way is to get real users to operate a system and see where they struggle. This is one such situation, for Peugeot Global, where I'll show how I did that.

Step One

My first task was to analyse the obvious cultural weaknesses of a direct translation from French to English. There were several problems, not the least of which was that the client was determined to use the word 'Mobility' in their design, a phrase that has a different connotation completely in Britain. These differences were quickly identified through both guerilla testing and simple heuristics, and a different labelling 'motion' applied for testing purposes.

Content comparison

Step Two

Secondly I devised a series of both closed and open card-sorting exercises to both test the existing structures and agreed naming conventions and also to see if there were any patterns in user perception that we hadn't predicted. As usual the user testing threw up a few wildcards, the term hybrid car, for instance, was simply not as familiar as first assumed, but if we substituted hybrid with 'environmentally friendly' there was no confusion at all. The more obvious results were used to refine the final testing prototype(s) so we could see if variants showed a similar pattern with different users.

card images

Step Three

The next step was behavioral testing and created a series of clickable prototypes to see how the proposed redesign(s) operated in real life, both in terms of navigation but also in interaction and engagement. These were iterative designs with user findings being used to refine solutions as the testing progressed. I set-up two rooms as impromptu testing labs and the tests were conducted over four days with me acting as Facilitator to a range of user type. The observers were a mixture of both system architects and developers and stakeholders from Peugeot UK. I have in the past been both observer lead and facilitator and both roles have their strengths, with the reactions of participants observers often proving to be illuminating. All sessions were recorded, both from an environmental view (the entire lab) and screen capture, for more deeper review after the testing.

Lab diagram

. - Facilitator (1) conducts the test and ensures candidates are at ease
- Candidate (2) Asked to talk out loud so that reactions and though processes could be recorded
- Workstation (3) All sessions were recorded on Camtasia
- Card Test (4) The user reorganised and renamed the existing structure in a logical - for them - layout
- CCTV (5) Sessions were recorded and observed in another room
- Screen (6) Large screen projection from the Testing Room
- Observers (7) Interested parties taking notes
- Notes (8) Observations reactions and points of concern that were recorded during the test
- Cards overhead (9) an overhead projection of the card sorting materials and exercise
- Room feed (10) a real-time view of the session broadcast from the camera (5)

Last Step

Finally, I analysed the tests, reporting on my findings, explaining how they had been reached and giving recommendations of how to proceed. The final site went live in 2012, with a much more consistent global branding style, yet was still flexible enough to allow national cultural requirements to be met.

What would I change in retrospect?

One thing, the luxury of time, the timings were part of the pitch that I had no control over and left little room for real detail in the final report.