A multi-level usability evaluation of mobile health applications: A case study

Objective: To report a methodological approach for the development of a usable mHealth application (app). Materials and methods: This work was guided by a 3-level stratified view of health information technology (IT) usability evaluation framework. We first describe a number of methodologies for operationalizing each level of the framework. Following the description of each methodology, we present a case study which illustrates the use of our preferred methodologies for the development of a mHealth app. At level 1 (user-task), we applied a card sorting technique to guide the information architecture of a mobile HIV symptom self-management app, entitled mVIP. At level 2 (user-task-system), we conducted a usability evaluation of mVIP in a laboratory setting through end-user usability testing and heuristic evaluation with informatics experts. At level 3 (user-task-system-environment), usability of mVIP was evaluated in a real-world setting following the use of the app during a 3-month trial. Results: The 3-level usability evaluation guided our work exploring in-depth interactions between the user, task, system, and environment. Integral to the findings from the 3-level usability evaluation, we iteratively refined the app's content, functionality, and interface to meet the needs of our intended end-users. Discussion and conclusion: The stratified view of the health IT usability evaluation framework is a useful methodological approach for the design, development, and evaluation of mHealth apps. The methodological recommendations for using the theoretical framework can inform future usability studies of mHealth apps.