Multimodal PTSD characterization via the StartleMart game

Computer games have recently shown promise as a diagnostic and treatment tool for psychiatric rehabilitation. This paper examines the potential of combining multiple modalities for detecting affective responses of patients interacting with a simulation built on game technology, aimed at the treatment of mental diagnoses such as post traumatic stress disorder (PTSD). For that purpose, we couple game design and game technology to create a game-based tool for exposure therapy and stress inoculation training that utilizes stress detection for the automatic profiling and potential personalization of PTSD treatments. The PTSD treatment game we designed forces the player to go through various stressful experiences while a stress detection mechanism profiles the severity and type of PTSD by analyzing the physiological responses to those in-game stress elicitors in two separate modalities: skin conductance (SC) and blood volume pulse (BVP). SC is often used to monitor stress as it is connected to the activation of the sympathetic nervous system (SNS). By including BVP into the model we introduce information about para-sympathetic activation, which offers a more complete view of the psycho-physiological experience of the player; in addition, as BVP is also modulated by SNS, a multimodal model should be more robust to changes in each modality due to particular drugs or day-to-day bodily changes. Overall, the study and analysis of 14 PTSD-diagnosed veteran soldiers presented in this paper reveals correspondence between diagnostic standard measures of PTSD severity and SC and BVP responsiveness and feature combinations thereof. The study also reveals that these features are significantly correlated with subjective evaluations of the stressfulness of experiences, represented as pairwise preferences. More importantly, the results presented here demonstrate that using the modalities of SC and BVP captures a more nuanced representation of player stress responses than using SC alone. We conclude that the results support the use of the simulation as a relevant treatment tool for stress inoculation training, and suggest the feasibility of using such a tool to profile PTSD patients. The use of multiple modalities appears to be key for an accurate profiling, although further research and analysis are required to identify the most relevant physiological features for capturing user stress.