Data-driven usability test scenario creation

van Eck, M. L., Markslag, E., Sidorova, N., Brosens-Kessels, A., & van der Aalst, W. M. P. (2019). Data-driven usability test scenario creation. In M. K. Lárusdóttir, M. Winckler, K. Kuusinen, P. Palanque, & C. Bogdan (Eds.), Human-Centered Software Engineering – 7th IFIP WG 13.2 International Working Conference, HCSE 2018, Revised Selected Papers (pp. 88-108). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11262 LNCS). Cham: Springer. DOI: 10.1007/978-3-030-05909-5_6


In this paper, we present a data-driven approach to enable the creation of evidence-based usability test scenarios. By utilising product usage data to create usability test scenarios, we aim to improve the reliability of the test results and to provide better insights into product usability. The approach consists of four elements: the collection of product usage data, the transformation of these data into logs of user activities, the creation of models of user behaviour, and the guided creation of usability test scenarios based on the models. We discuss the challenges that can be encountered when applying this approach based on our experiences with two case studies in product development. We have created a prototype scenario planning tool and performed a preliminary evaluation of the tool with usability engineers working at Philips Healthcare. The evaluation shows that tool-supported evidence-based usability test creation would be valuable in their daily work.

Leave a Reply