Useful Research about Usability Testing

Question 1: How many users per user group for Usability Testing?

Answer: There are some researches for Sample Size –

1. Five users will uncover approximately 80% of the usability problems in a product (Virzi, 1992 and Nielsen 1993).

2. Spool and Schroeder (2001) report that testing five users revealed only 35% of the problems.

3. Faulkner (2003) found that:

Testing five users revealed an average of 85% of usability problems, but percentage ranged from nearly 100% down to only 55%.

10 participants help ensure that 80% of usability problems will be found.

Question 2: Should usability test participants ‘think Aloud’ or should they report their experience after the task is completed?

Answer: Retrospective vs. Concurrent think aloud protocols – Testing the usability of an online library catalogue. Van Den Haak, M.L., 2003.

“Both Technique reveal comparable sets of Usability Problems. Participants worked more slowly when thinking aloud, suggesting that when task time is an issue, alternatives such as retrospective reposts should be used.”

Questions 3: How does having a facilitator present affect usability test performance?

Answer: Information search in the laboratory and on the web: with and without experimenter – schtle – Mecklenbeck, M., & Huber, O. 2003.

“Users spend about twice as long and clicked three times as many links when facilitator is present. Facilitated testing environments may result in artificially diligent attempts at completing a task relative to non-facilitated tests.”

Question 4: Which is the best method for testing mobile application: Laboratory or field testing

Answer: Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing. Anne Kaikkonen, Aki Kekalainen, Mihael Cankar, Titti Kallio, and Anu Kankainen. 2005.

“When testing a user interface of a mobile app, field testing may not necessarily be the best place; mostly because it is more time consuming than the lab test.

Testing in the field requires double the time in comparison to the laboratory. In the field you can run half of the tests per day you run in the laboratory.

In a field test, running a pre-test or a pilot is critical: there are so many details that can go wrong, and you really need to check that everything is working correctly.”