Automatic evaluation of facial nerve paralysis by dual-path LSTM with deep differentiated network

Facial nerve paralysis adversely affects the patients' mental and physical health, and several existing evaluation methods of facial paralysis are put forward based on the static facial asymmetry. However, these traditional methods still suffer from two drawbacks: (1) the facial movement information is always being ignored, which plays important roles for facial paralysis analysis; (2) the shallow machine learning models have their limitations on extracting useful facial features for the evaluation of facial paralysis. To solve these problems, we present dual-path LSTM with deep differentiated network (DP-LSTM-DDN) to evaluate the severity of facial paralysis automatically. The key idea behind DP-LSTM-DDN is that the diagnosis results are sensitive to the facial asymmetry and the patterns of facial muscular movements when the patients were doing the diagnostic facial actions. Therefore, we design a deep differentiated network to analyze the difference between two sides of patients' faces. Furthermore, since the involved facial regions are as important as the whole face for the diagnostic of facial paralysis analysis, we propose a dual-path LSTM network to extract both global and local facial movement features. Then these extracted high level representations are fused for the final evaluation of facial paralysis. The experimental results have verified the better performance of DP-LSTM-DDN compared with the state-of-the-art methods. (C) 2020 Elsevier B.V. All rights reserved.