Chinese Sign Language Recognition Based on an Optimized Tree-Structure Framework

Chinese Sign Language (CSL) subword recognition based on surface electromyography (sEMG), accelerometer (ACC), and gyroscope (GYRO) sensors was explored in this paper. In order to fuse effectively the information of these three kinds of sensors, the classification abilities of sEMG, ACC, GYRO, and their combinations in three common sign components (one or two handed, hand orientation, and hand amplitude) were evaluated first and then an optimized tree-structure classification framework was proposed for CSL subword recognition. Eight subjects participated in this study and recognition experiments under different testing conditions were implemented on a target set consisting of 150 CSL subwords. The proposed optimized tree-structure classification framework based on sEMG, ACC, and GYRO obtained the best performance among seven different testing conditions with single sensor, paired-sensor fusion, and three-sensor fusion, and the overall recognition accuracies of 94.31% and 87.02% were obtained for 150 CSL subwords in a user-specific test and user-independent test, respectively. Our study could lay a basis for the implementation of large-vocabulary sign language recognition system based on sEMG, ACC, and GYRO sensors.