NASA RTLX as a Novel Assessment for Determining Cognitive Load and User Acceptance of Expert and User-Based Evaluation Methods Exemplified Through a mHealth Diabetes Self-Management Application Evaluation
Many diabetes patients utilize mobile health applications for their personalized self-management needs, but a majority of these systems have deficient usability. To assess the usability, different evaluation methods can be used. Two such methods are the expert-based cognitive walkthrough (CW) and user-based think aloud (TA) which have different advantages and disadvantages and can lead to the detection of many types of usability problems. There is, however, a lack of research on how the evaluators experience performing these different methods; an important aspect to include due to its possible implications for the evaluation overall. In this paper the focus was on assessing the evaluators' cognitive load and method acceptance through an instrument novel for this particular purpose, the NASA RTLX, and through in-depth interviews. Six usability experts and 6 diabetes patients performed the evaluations where 18 usability problems were found with the CW and 20 with the TA and with system usability (SUS) scores of 23.75 and 59.58. Both methods had similarly high cognitive loads: the CW with a RTLX score of 56.11, and the TA of 53.47 and both were experienced as cognitively demanding. The results indicate the potential importance of these dimensions for decision-making purposes between different evaluation methods.
IOS Press, Inc.
6751 Tepper Drive
Clifton, VA 20124
Tel.: +1 703 830 6300
Fax: +1 703 830 2300 email@example.com
(Corporate matters and books only) IOS Press c/o Accucoms US, Inc.
For North America Sales and Customer Service
West Point Commons
Lansdale PA 19446
Tel.: +1 866 855 8967
Fax: +1 215 660 5042 firstname.lastname@example.org