Revisiting Heuristic Evaluation Methods to Improve the Reliability of Findings
2014 (English)Conference paper (Refereed) PublishedAlternative title
Omprövning av Heuristiska utvärderingsmetoder i syfte att förbättra dess reliabilitet (Swedish)
The heuristic evaluation (HE) method is one of the most common in the suite of tools for usability evaluations because it is a fast, inexpensive and resource-efficient process in relation to the many usability issues it generates. The method emphasizes completely independent initial expert evaluations. Inter-rater reliability and agreement coefficients are not calculated. The variability across evaluators, even dual domain experts, can be significant as is seen in the case study here. The implications of this wide variability mean that results are unique to each HE, results are not readily reproducible and HE research on usability is not yet creating a uniform body of knowledge. We offer recommendations to improve the science by incorporating selected techniques from qualitative research: calculating inter-rater reliability and agreement scores, creating a codebook to define concepts/categories and offering crucial information about raters' backgrounds, agreement techniques and the evaluation setting.
Place, publisher, year, edition, pages
Istanbul: IOS Press , 2014.
Heuristic Evaluation, eHealth, mHealth, Usability Research Methods
Human Aspects of ICT Medical Laboratory and Measurements Technologies
IdentifiersURN: urn:nbn:se:bth-6616DOI: 10.3233/978-1-61499-432-9-930Local ID: oai:bth.se:forskinfoA0CF3229ADA55F8AC1257D5B0021FF02ISBN: 978-1-61499-432-9OAI: oai:DiVA.org:bth-6616DiVA: diva2:834138
Studies in Health Technology and Informatics, IOS Press, indexed in Medline (PubMed)2014-09-242014-09-222016-10-11Bibliographically approved