Measuring cognitive load during simulation-based psychomotor skills training: sensitivity of secondary-task performance and subjective ratings

As interest in applying cognitive load theory (CLT) to the study and design of pedagogic and technological approaches in healthcare simulation grows, suitable measures of cognitive load (CL) are needed. Here, we report a two-phased study investigating the sensitivity of subjective ratings of mental effort (SRME) and secondary-task performance (signal detection rate, SDR and recognition reaction time, RRT) as measures of CL. In phase 1 of the study, novice learners and expert surgeons attempted a visual-monitoring task under two conditions: single-task (monitoring a virtual patient's heart-rate) and dual-task (tying surgical knots on a bench-top simulator while monitoring the virtual patient's heart-rate). Novices demonstrated higher mental effort and inferior secondary-task performance on the dual-task compared to experts (RRT 1.76 vs. 0.73, p = 0.012; SDR 0.27 vs. 0.97, p < 0.001; SRME 7.75 vs. 2.80, p < 0.001). Similarly, secondary task performance deteriorated from baseline to dual-task among novices (RRT 0.63 vs. 1.76 s, p < 0.006 and SDR 1.00 vs. 0.27, p < 0.001), but not experts (RRT 0.63 vs. 0.73 s, p = 0.124 and SDR 1.00 vs. 0.97, p = 0.178). In phase 2, novices practiced surgical knot-tying on the bench top simulator during consecutive dual-task trials. A significant increase in SDR (F-(9,F-63) = 6.63, p < 0.001, f = 0.97) and decrease in SRME (F-(9,F-63) = 9.39, p < 0.001, f = 1.04) was observed during simulation training, while RRT did not change significantly (F-(9,F-63) = 1.18, p < 0.32, f = 0.41). The results suggest subjective ratings and dual-task performance can be used to track changes in CL among novices, particularly in early phases of simulation-based skills training. The implications for measuring CL in simulation instructional design research are discussed.