Detecting naturalistic expression of emotions using physiological signals while playing video games

Affective gaming has been an active research field recently. This is due to the importance of the player's emotions while playing computer games. Emotions can be detected from various modalities such as facial, voice, and physiological signals. In this study, we evaluate an XGBoost ensemble method and deep neural network for detecting naturalistic expressions of emotions of video game players using physiological signals. Physiological data was collected from twelve participants while playing PUBG mobile game. Both Discrete and dimensional emotion models were evaluated. We evaluated the performance of classification models using individual physiological channels and a fusion of these channels. A comparison between user-dependent, and user-independent is also provided. Our results indicated that the use of the dimensional valence and arousal model can provide more accurate accuracy than the discrete emotion model. The results also showed that ECG features and a fusion of features from all physiological channels provide the highest affect detection accuracy. Our deep neural network model based on user-dependent model achieved the highest accuracy with 77.92% and 78.58% of detecting valence, and arousal respectively using a fusion of features. The user-independent models were not feasible, presumably due to strong individual differences of physiological responses.

Automated summary

Affective gaming has recently become an active research field focusing on detecting emotions of players during computer games. The study evaluated an XGBoost ensemble method and deep neural network for detecting naturalistic expressions of emotions using physiological signals. Results indicated that using a dimensional valence and arousal model provided more accurate accuracy than a discrete emotion model, with ECG features and a fusion of features from all physiological channels showing the highest affect detection accuracy. The deep neural network model based on a user-dependent model achieved the highest accuracy, while user-independent models were not feasible due to strong individual differences in physiological responses.