期刊
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING
卷 27, 期 6, 页码 1149-1159出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TNSRE.2019.2913400
关键词
Mental workload; deep learning; feature fusion; convolutional neural network; EEG
资金
- National Key Research and Development Program of China [2018YFB2003501]
- National Natural Science Foundation of China [61472216]
People's mental workload profoundly affects their work efficiency and health. Mental workload assessment can be used to effectively avoid serious accidents caused by excessive mental workload. Both electroencephalogram (EEG) spectral features and its temporal features have proven to be useful in addressing this problem. The fusion of the two types of features can provide rich distinguishing information for improving mental workload assessment. Benefiting from the progress of deep learning, this study proposes the two-stream neural networks (TSNN) for fusing the two types of EEG features. Compared with hand-crafted features, the TSNN can learn and fuse EEG features from the spectral and temporal dimensions automatically without prior knowledge. The TSNN includes a spectral stream and a temporal stream. Each stream consists of a convolutional neural network (CNN) and a temporal convolutional network (TCN) to learn spectral or temporal features from EEG topographic maps. To fuse the learned spectral and temporal information, we concatenate the output of the two streams prior to the fully connected layer. EEG data were collected from 17 subjects who performed n-back tasks with easy, medium, and hard difficulty levels, leading to a three-class mental workload classification. The results show that the TSNN achieves an average accuracy of 91.9%, which is a significant improvement over baseline classifiers based on hand-crafted features. The TSNN also outperforms state-of-the-art deep learning methods developed for EEG classification. The results indicate that the proposed structure is promising for fusing spectral and temporal features for mental workload assessment. In addition, it provides a high-precision approach for potential applications during cognitive activities.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据