Should You Engage Stakeholders in Usability Testing?

You are here

Often it is more efficient to simply leave one person who is in charge of usability testing and let the rest of the team stay in the office. However, when team members are involved in the usability process and see all the issues for themselves at live user test sessions, the effect appears much more powerful.

Having team members at usability sessions suggests many benefits:

Credibility. After seeing you derive your insights, they are more likely to believe the usability findings in reports you provide.

Buy-in. Besides inviting team members for observation, it may be of use to engage them in a debriefing, discussing what happened during the test sessions, participating in drawing some early conclusions etc. When people take part in the analysis, it makes them more pliable to accept the recommendations.

Memorability. Findings read in a lengthy report or seen only in bullet points are hard to remember. On the other hand, the findings from your personal experience in observing some user sessions are much easier to remember.

Empathy. Seeing users suffer from your poor design inevitably motivates you to make it better. Moreover, the team members won’t be able to attribute some of their project inconsistences to ignorant users when they hear these users make perfectly reasonable and articulate for a design to suit their needs better.

Fewer design mistakes. Designers and developers who have seen the actual customers will be less likely to incorporate some of their design ideas obviously not working for these users. Besides, the good raw UI will need fewer fixes after the next user testing round.

Most of your team will have no time to attend every user session. It’s okay, but people’s failure to witness the complete study may also result in some problems.

Risks of Partial Observations:

Memory bias. Our memory is fairly to biases and is also quite poor at remembering some abstractions. The best point in observing the user sessions is remembering usability issues better due to the first-hand perception. However, the danger is you are likely to remember the usability issues seen for yourself far much better the ones only read about on the report of the full study.

Entertainment bias. A variation of memory bias, entertainment bias suggests you’ll better remember some particularly striking and outspoken users than quiet ones. Therefore, to enhance the site or product’s business value you should test representative customers rather than entertaining ones.

Empathy bias. There’s also a chance that managers, designers and developers will subconsciously feel more likely to cater to the users they've seen rather than to meet other users’ needs they’ve just heard about, which is equally important.

Premature conclusions. To get a solid insight in the overall usability testing 5 users is enough. But testing 1-2 users to define actual trends and patterns is not. Anyone seeing just a few users may misidentify the key usability issues and get no insight enough to correctly analyze the solutions.

The problems listed above are inevitable, but once you become aware of them, you are able to alleviate them and warn other stakeholders about these. In any case, practice shows that the downsides are always outweighed by the advantages of inviting executives and colleagues to attend as many user sessions as they can.