Review
Chemistry, Analytical
Janis Peksa, Dmytro Mamchur
Summary: This paper gives a comprehensive overview of the current state-of-the-art in brain-computer interfaces (BCI). It introduces the basic principles and widely used platforms of BCIs. It examines the various components of a BCI system, including hardware, software, and signal processing algorithms. It discusses the current research trends and potential future applications of BCI technology in medical, educational, and other fields, as well as the challenges that need to be addressed for widespread adoption.
Article
Computer Science, Artificial Intelligence
Siddharth, Tzyy-Ping Jung, Terrence J. Sejnowski
Summary: This research applies novel deep-learning-based methods to analyze and evaluate four publicly available multi-modal emotion datasets containing bio-sensing and video data. The algorithms outperform previous studies in emotion classification and set benchmarks for new datasets. The research also overcomes inconsistencies between datasets using transfer learning and proposes a new technique for identifying salient brain regions corresponding to affective states.
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING
(2022)
Article
Psychology, Multidisciplinary
Mingxing Liu
Summary: This paper presents an in-depth analysis of the emotional classification of EEG neurofeedback interactive electronic music compositions using a multi-brain collaborative brain-computer interface (BCI). It explores the design and performance of sound visualization in an interactive format and proposes a specific mapping model for the conversion of sound to visual expression. Furthermore, it introduces a dynamic brain network approach for analyzing EEG signals and achieving emotion recognition.
FRONTIERS IN PSYCHOLOGY
(2022)
Article
Chemistry, Analytical
Mahrad Ghodousi, Jachin Edward Pousson, Valdis Bernhofs, Inga Griskova-Bulanova
Summary: This study evaluated different features to discriminate emotions in music performance and aimed to develop a brain-computer music interface system. Power spectra and connectivity features were extracted from EEG signals and their capabilities in emotion detection were compared using Support Vector Machine. Results showed that connectivity features had higher accuracy and shorter processing time, making them potential candidates for the development of a real-time brain-computer music interface system.
Review
Chemistry, Analytical
Kais Belwafi, Sofien Gannouni, Hatim Aboalsamh
Summary: BCI systems have a wide range of applications in restoring capabilities for people with severe motor disabilities, with a growing number of systems being developed. There is a significant interest in implementing BCIs on portable platforms, with smaller size, faster loading times, lower cost, fewer resources, and lower power consumption compared to full PCs.
Article
Engineering, Electrical & Electronic
Dongrui Wu, Bao-Liang Lu, Bin Hu, Zhigang Zeng
Summary: A brain-computer interface (BCI) allows direct communication between a user and a computer through the central nervous system. An affective BCI (aBCI) monitors and regulates the emotional state of the brain, which has various applications in human cognition, communication, decision-making, and health. This tutorial provides a comprehensive and up-to-date guide on aBCIs, covering basic concepts, components of a closed-loop aBCI system, representative applications, and challenges and opportunities in aBCI research and applications.
PROCEEDINGS OF THE IEEE
(2023)
Article
Computer Science, Information Systems
Ruoyu Du, Shujin Zhu, Huangjing Ni, Tianyi Mao, Jiajia Li, Ran Wei
Summary: During the COVID-19 pandemic, young people are using multimedia content more frequently to communicate on Internet platforms, and music, as psychological support, plays an important role in emotional self-regulation. This paper focuses on the valence and arousal classification of Chinese ancient-style music-evoked emotion and proposes a hybrid model to achieve this classification. The experimental results show that the proposed method has high accuracy in the emotion classification task.
MULTIMEDIA TOOLS AND APPLICATIONS
(2023)
Article
Psychology, Multidisciplinary
Xiaoqing Gu, Yiqing Fan, Jie Zhou, Jiaqun Zhu
Summary: An optimized projection and Fisher discriminative dictionary learning (OPFDDL) model is proposed in this study to efficiently exploit the specific discriminative information of each frequency band in EEG signals for emotion recognition.
FRONTIERS IN PSYCHOLOGY
(2021)
Article
Computer Science, Artificial Intelligence
Eva Zangerle, Chih-Ming Chen, Ming-Feng Tsai, Yi-Hsuan Yang
Summary: This study analyzes the connection between users' emotional states and their musical choices, finding that affective information has a significant impact on music recommendation. Different ranking strategies are proposed based on emotional information and latent features, with those incorporating affective information and leveraging hashtags outperforming others in capturing context-specific preferences.
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING
(2021)
Article
Biology
Jose M. Macias-Macias, Juan A. Ramirez-Quintana, Mario I. Chacon-Murguia, Alejandro A. Torres-Garcia, Luis F. Corral-Martinez
Summary: In this paper, a method called CapsK-SI, which uses statistical features and a Capsule Neural Network, is proposed to classify imagined phonemes and words in speech imagery signals. The accuracy of the classification is high for different categories, reaching over 90% for most cases. Furthermore, brain maps are generated to represent brain activity in the production of certain speech signals.
COMPUTERS IN BIOLOGY AND MEDICINE
(2023)
Article
Multidisciplinary Sciences
Badar Almarri, Sanguthevar Rajasekaran, Chun-Hsi Huang
Summary: This paper introduces a subject-independent emotion recognition framework that reduces subject-to-subject variability by adequate preprocessing, transforming, and feature extraction prior to analyzing emotion data. By utilizing unsupervised algorithms and support vector machine, the study outperforms other subject-independent studies in accurately classifying human affection based on EEG benchmarks.
Article
Neurosciences
Mari Tervaniemi, Tommi Makkonen, Peixin Nie
Summary: The study found that there were differences in participants' emotion ratings between listening to music at home and in the laboratory, and cortisol levels were generally lower at home. However, the modulatory effects of music on cortisol levels were not different between the two environments.
Article
Engineering, Electrical & Electronic
Kranti S. Kamble, Joydeep Sengupta, Kranti Kamble
Summary: This study compares machine learning-based algorithms with conventional machine learning algorithms for emotion recognition from EEG signals. The results show that machine learning-based algorithms outperform conventional machine learning algorithms in multiclass emotion recognition.
IEEE SENSORS JOURNAL
(2022)
Article
Chemistry, Analytical
Sofien Gannouni, Kais Belwafi, Arwa Aledaily, Hatim Aboalsamh, Abdelfettah Belghith
Summary: It proposes an original framework for usability testing based on emotion detection using EEG signals, which can significantly affect software production and user satisfaction. The framework includes a recurrent neural network algorithm as a classifier, a feature extraction algorithm based on event-related desynchronization and event-related synchronization analysis, and a new method for selecting EEG sources adaptively for emotion recognition.
Article
Chemistry, Multidisciplinary
Awf Abdulrahman, Muhammet Baykara, Talha Burak Alakus
Summary: Emotion is a voluntary or involuntary reaction to external factors, expressed through actions. Recent studies have shown that emotion analysis based on EEG signals is reliable. In this study, a deep learning model was proposed to analyze EEG signals and obtain statistical features for classification.
APPLIED SCIENCES-BASEL
(2022)
Article
Engineering, Biomedical
Ze Wang, Chi Man Wong, Agostinho Rosa, Tao Qian, Tzyy-Ping Jung, Feng Wan
Summary: This study proposes a new perspective, the time-frequency-joint representation, which synchronizes SSVEP signals corresponding to different stimuli and emphasizes common components. By extracting and transferring these common components, it improves the performance of SSVEP recognition and reduces the calibration time, facilitating real-world applications of BCIs.
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
(2023)
Article
Computer Science, Artificial Intelligence
Wen-Chi Chou, Hsiao-Ching She, Tzyy-Ping Jung
Summary: Despite advances in neuroscience, the mechanisms of relational reasoning in optical image formation tasks and their relationship with task difficulty remain unclear. This study explores the underlying brain dynamics involved in optical image formation tasks of different difficulty levels. Single mirror relational reasoning tasks showed higher response accuracy and shorter latency compared to single convex lens tasks of high difficulty. Stronger brain dynamics and coordination, including frontal midline theta and parietal alpha power, are crucial for solving tasks with higher difficulty.
INTERNATIONAL JOURNAL OF NEURAL SYSTEMS
(2023)
Article
Engineering, Biomedical
Jin Han, Minpeng Xu, Xiaolin Xiao, Weibo Yi, Tzyy-Ping Jung, Dong Ming
Summary: This study developed a high-speed BCI system with more than 200 targets, encoded by a combination of electroencephalography features. The system achieved high accuracy and information transfer rate in offline and online experiments, showing promise for extending BCI's application scenarios.
JOURNAL OF NEURAL ENGINEERING
(2023)
Review
Computer Science, Information Systems
Congying He, Yu-Yi Chen, Chun-Ren Phang, Cory Stevenson, I-Ping Chen, Tzyy-Ping Jung, Li-Wei Ko
Summary: Wireless electroencephalography (EEG) systems have gained increasing attention, with a growing number of articles and a rising proportion compared to general EEG publications. This signifies the accessibility and potential of wireless EEG systems. This review explores the development and applications of wearable and wireless EEG systems, evaluating the specifications of major systems in the market and discussing their use in consumer, clinical, and research settings. It suggests that low-price and convenience are important for consumers, FDA or CE-certified systems are suitable for clinical settings, and high-density channel devices are crucial for laboratory research. This article provides an overview and serves as a guide for the current state and future development of wireless EEG systems.
IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS
(2023)
Article
Engineering, Biomedical
Ruixin Luo, Minpeng Xu, Xiaoyu Zhou, Xiaolin Xiao, Tzyy-Ping Jung, Dong Ming
Summary: SAME is an effective method for increasing the calibration data in SSVEP-BCIs, thus significantly improving the performance of eTRCA and TDCA. Combined with SAME, the average accuracy of eTRCA and TDCA can be increased by about 12% and 3% respectively, even with limited calibration data. SAME also enables eTRCA and TDCA to work well with just one calibration trial, achieving an average accuracy >90% for the Benchmark dataset and >70% for the BETA dataset.
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
(2023)
Article
Engineering, Biomedical
Kuan-Jung Chiang, Steven Dong, Chung-Kuan Cheng, Tzyy-Ping Jung
Summary: This study investigated the associations between memory workload and EEG during participants' typical office tasks on a single-monitor and dual-monitor arrangement. The study found significant differences in EEG characteristics between high and low memory workload states, consistent across all participants. The effectiveness of using EEG analysis in real-world neuroergonomic studies was demonstrated.
JOURNAL OF NEURAL ENGINEERING
(2023)
Article
Computer Science, Artificial Intelligence
Kangning Wang, Shuang Qiu, Wei Wei, Yukun Zhang, Shengpei Wang, Huiguang He, Minpeng Xu, Tzyy-Ping Jung, Dong Ming
Summary: In this study, a 4-target BCI system based on SSVEP was built to assist people with disabilities. EEG and EOG data were recorded from 18 subjects during a 90-min continuous task, and a multimodal vigilance estimating network, MVENet, was proposed to estimate the vigilance state of BCI users. Experimental results showed that the network achieved better performance than the compared methods, demonstrating the feasibility and effectiveness of our methods for estimating the vigilance state of BCI users.
EXPERT SYSTEMS WITH APPLICATIONS
(2023)
Article
Engineering, Electrical & Electronic
Shun Liu, Chi Man Wong, Xucheng Liu, Hongtao Wang, Anastasios Bezerianos, Yu Sun, Tzyy-Ping Jung, Feng Wan
Summary: This study examined the neural mechanism of driver fatigue by investigating the cross-frequency coupling of slow and fast oscillations in a multilayer brain network. It was found that the coupling in the fatigue state was enhanced, particularly in beta-gamma coupling and the frontal, frontal pole, and parietal regions. Significant differences were also observed in the topology of the multilayer brain network between vigilant and fatigue states, including increased global and local efficiencies in the fatigue state. A graph neural network (GNN) was developed to detect fatigue with high accuracy (96.23%) by imitating the features of the within-frequency subnetworks diffused through cross-frequency coupling. This research provides insights into neural coordination in driver fatigue and can contribute to reducing traffic accidents.
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT
(2023)
Article
Engineering, Biomedical
Haoyin Xu, Sheng-Hsiou Hsu, Masaki Nakanishi, Yufan Lin, Tzyy-Ping Jung, Gert Cauwenberghs
Summary: Visual stimuli design in brain-computer interfaces based on visual evoked potentials (VEPs) plays a crucial role. This study comprehensively compared different combinations of stimulus parameters and their impact on decoding accuracy and subject comfort. The results revealed a trade-off relationship between decoding accuracy and subjective comfort level, with code-modulated VEPs (c-VEPs) being recommended for achieving reliable decoding accuracy while maintaining a reasonable level of comfortability.
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING
(2023)
Article
Engineering, Biomedical
Lubin Meng, Xue Jiang, Jian Huang, Zhigang Zeng, Shan Yu, Tzyy-Ping Jung, Chin-Teng Lin, Ricardo Chavarriaga, Dongrui Wu
Summary: This paper proposes a narrow period pulse-based poisoning attack on EEG-based BCIs, which makes adversarial attacks easier to implement. By injecting poisoning samples into the training set, dangerous backdoors can be created in the machine learning model. Test samples with the backdoor key will be classified into the target class specified by the attacker. The distinguishing feature of this approach is that the backdoor key does not need to be synchronized with the EEG trials, making it very easy to implement. The effectiveness and robustness of the backdoor attack approach is demonstrated, raising a critical security concern for EEG-based BCIs and calling for urgent attention to address it.
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING
(2023)
Article
Engineering, Biomedical
Chi Man Wong, Ze Wang, Boyu Wang, Agostinho Rosa, Tzyy-Ping Jung, Feng Wan
Summary: A phase difference constrained CCA (pdCCA) method is proposed to improve the recognition performance of multi-frequency-modulated SSVEP. Experimental results show that the pdCCA-based method significantly outperforms the traditional CCA method in terms of recognition accuracy.
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING
(2023)
Article
Engineering, Biomedical
Masaki Nakanishi, Annalise Miner, Tzyy-Ping Jung, Jennifer Graves
Summary: Afferent and efferent visual dysfunction are important features of multiple sclerosis, and visual outcomes have been proven to be reliable biomarkers of the overall disease state. However, precise measurement of these dysfunctions is limited to specialized facilities, and the measurements are currently unavailable in acute care facilities. In this study, a novel moving mfSSVEP stimulus was developed to evaluate afferent and efferent visual function simultaneously on a mobile platform.
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING
(2023)
Proceedings Paper
Computer Science, Artificial Intelligence
Feng He, Jieyu Wu, Xiaolin Xiao, Runyuan Gao, Weibo Yi, Yuanfang Chen, Minpeng Xu, Tzyy-Ping Jung, Dong Ming
Summary: This study developed a 10-command SSVEP-BCI system in mixed reality using Hololens2 and investigated the impact of two stimulus colors (white and red) on four monochrome backgrounds (green, blue, white, and black). The results showed that both stimulus and background colors significantly affected BCI performance, and color contrast ratio was an important criterion for optimization.
HUMAN BRAIN AND ARTIFICIAL INTELLIGENCE, HBAI 2022
(2023)
Article
Engineering, Biomedical
Wei-Che Lin, Wei-Jen Chen, Yueh-Sheng Chen, Hsing-Yi Liang, Cheng-Hsien Lu, Yuan-Pin Lin
Summary: This study proposes an EEG-driven machine-learning scenario for automatically assessing impulse control disorders (ICDs) comorbidity in patients with Parkinson's disease (PD). The results showed that the SVM pipeline differentiated subjects with ICD from subjects with PD with an accuracy of 66.3% and the SVR pipeline yielded significantly higher severity scores for the ICD group than for the PD group. This demonstration may facilitate deploying a wearable computer-aided diagnosis system to assess the risk of DA-triggered cognitive comorbidity in patients with PD in their daily environment.
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING
(2023)
Article
Engineering, Biomedical
Shang-You Yang, Yuan-Pin Lin
Summary: Wearable low-density dry EEG headsets have broad applications in decoding brain activity and brain-triggered interaction for healthy individuals. However, movement artifacts present challenges to their validity outside laboratory settings. This study explored the effectiveness of hardware and software solutions for low-density and dry EEG recordings in non-tethered settings. Results showed that active-electrode designs effectively corrected movement artifacts for dry electrodes, while the ASR pipeline was compromised by limited electrodes. This suggests that lightweight, minimally obtrusive dry EEG headsets should at least include an active-electrode infrastructure to maintain their validity and applicability in real-world scenarios.
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING
(2023)