Abstract
It is widely assumed that 5 participants suffice for usability testing. In this study, 60 users were tested and random sets of 5 or more were sampled from the whole, to demonstrate the risks of using only 5 participants and the benefits of using more. Some of the randomly selected sets of 5 participants found 99% of the problems; other sets found only 55%. With 10 users, the lowest percentage of problems revealed by any one set was increased to 80%, and with 20 users, to 95%.
Article PDF
Similar content being viewed by others
References
Grosvenor, L. (1999).Software usability: Challenging the myths and assumptions in an emerging field. Unpublished master’s thesis, University of Texas, Austin.
Hudson, W. (2001). How many users does it take to change a website?SIGCHI Bulletin May/June 2001. Retrieved April 15, 2003 from http://www.acm.org/sigchi/bulletin/2001.3/mayjun01.pdf.
Landauer, T. K., & Nielsen, J. (1993). A mathematical model of the finding of usability problems.Interchi ’93, ACM Computer-Human Interface Special Interest Group.
Nielsen, J. (1993).Usability engineering. Boston: AP Professional.
Nielsen, J. (2000, March).Why you only need to test with 5 users: Alertbox. Retrieved April 15, 2003 from http://www.useit.com/ alertbox/20000319.html.
Perfetti, C., & Landesman, L. (2002).Eight is not enough. Retrieved April 14, 2003 from http://world.std.com/~uieweb/Articles/eight_is_ not enough.htm.
Reason, J. (1997).Managing the risks of organizational accidents. Brookfield, VT: Ashgate.
Spool, J., &Schroeder, W. (2001). Testing web sites: Five users is nowhere near enough. InCHI 2001 Extended Abstracts (pp. 285–286). New York: ACM Press.
Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough?Human Factors,34, 457–468.
Wentworth, R. J. (1996, March).Group chairman’s factual report (American Airlines Flight 965). Washington, DC: National Transportation Safety Board, Office of Aviation Safety.
Woolrych, A., &Cockton, G. (2001). Why and when five test users aren’t enough. In J. Vanderdonckt, A. Blandford, & A. Derycke (Eds.),Proceedings of IHM-HCI 2001 Conference: Vol. 2 (pp. 105–108). Toulouse, France: Cépadèus.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Faulkner, L. Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, & Computers 35, 379–383 (2003). https://doi.org/10.3758/BF03195514
Received:
Accepted:
Issue date:
DOI: https://doi.org/10.3758/BF03195514
