An integrated approach for medical abnormality detection using deep patch convolutional neural networks

Computer-aided detection of abnormalities in medical images has clinical significance but remains a challenging research topic. Unlike object detection in natural images, the detection of medical abnormalities is unique because they often locate in tiny local regions within a high-resolution medical image. Traditional machine learning approaches build sliding windowbased detectors with manual features and are therefore limited on both efficiency and accuracy. Recent advances in deep learning shed new light on this problem, but applying it to medical image analysis faces challenges including insufficient data. Training deep convolutional neural networks (CNNs) directly on high-resolution images requires image compression at the input layer, which leads to the loss of information that is essential for medical abnormality detection. Therefore, instead of training on full images, the proposed approach first fine-tunes the pre-trained deep CNNs on image patches centered at medical abnormalities and then integrates them with class activation mappings and region proposal networks for building abnormality detectors. The deep patch classifier has been tested on a mammogram data set and achieved an overall classification accuracy of 92.53%, compared to 81.55% by a traditional approach using manual features. The integrated detector has been tested on an ultrasound liver image data set for abnormality detection and achieved an average precision of 0.60, outperforming both the sliding window-based approach of 0.16 and a deep learning YOLO model of 0.51. These validations suggest that the integrated approach has great potential for assisting doctors in detecting abnormalities from medical images.