A knowledge-driven feature learning and integration method for breast cancer diagnosis on multi-sequence MRI

Background: The classification of benign versus malignant breast lesions on multi-sequence Magnetic Resonance Imaging (MRI) is a challenging task since breast lesions are heterogeneous and complex. Recently, deep learning methods have been used for breast lesion diagnosis with raw image input. However, without the guidance of domain knowledge, these data-driven methods cannot ensure that the features extracted from images are comprehensive for breast cancer diagnosis. Specifically, these features are difficult to relate to clinically relevant phenomena. Purpose: Inspired by the cognition process of radiologists, we propose a Knowledge-driven Feature Learning and Integration (KFLI) framework, to discriminate between benign and malignant breast lesions using Multi-sequences MRI. Methods: Starting from sequence division based on characteristics, we use domain knowledge to guide the feature learning process so that the feature vectors of sub-sequence are constrained to lie in characteristic-related semantic space. Then, different deep networks are designed to extract various sub-sequence features. Furthermore, a weighting module is employed for the integration of the features extracted from different subsequence images adaptively. Results: The KFLI is a domain knowledge and deep network ensemble, which can extract sufficient and effective features from each sub-sequence for a comprehensive diagnosis of breast cancer. Experiments on 100 MRI studies have demonstrated that the KFLI achieves sensitivity, specificity, and accuracy of 84.6%, 85.7% and 85.0%, respectively, which outperforms other state-of-the-art algorithms.