Multi-modality CADx: ROC Study of the Effect on Radiologists' Accuracy in Characterizing Breast Masses on Mammograms and 3D Ultrasound Images

Rationale and Objectives. To investigate the effect of a computer-aided diagnosis (CADx) system on radiologists' performance in discriminating malignant and benign masses on mammograms and three-dimensional (313) ultrasound (US) images. Materials and Methods. Our dataset contained mammograms and 3D US volumes from 67 women (median age, 5 1; range: 27-86) with 67 biopsy-proven breast masses (32 benign and 35 malignant). A CADx system was designed to automatically delineate the mass boundaries on mammograms and the US volumes, extract features, and merge the extracted features into a multi-modality malignancy score. Ten experienced readers (subspecialty academic breast imaging radiologists) first viewed the mammograms alone, and provided likelihood of malignancy (LM) ratings and Breast Imaging and Reporting System assessments. Subsequently, the reader viewed the US images with the mammograms, and provided LM and action category ratings. Finally, the CADx score was shown and the reader had the opportunity to revise the ratings. The LM ratings were analyzed using receiver-operating characteristic (ROC) methodology, and the action category ratings were used to determine the sensitivity and specificity of cancer diagnosis. Results. Without CADx, readers' average area under the ROC curve, A(z), was 0.93 (range, 0.86-0.96) for combined assessment of the mass on both the US volume and mammograms. With CADx, their average A(z), increased to 0.95 (range, 0.91-0.98), which was borderline significant (P = .05). The average sensitivity of the readers increased from 98% to 99% with CADx, while the average specificity increased from 27% to 29%. The change in sensitivity with CADx did not achieve statistical significance for the individual radiologists, and the change in specificity was statistically significant for one of the radiologists. Conclusions. A well-trained CADx system that combines features extracted from mammograms and US images may have the potential to improve radiologists' performance in distinguishing malignant from benign breast masses mid making decisions about biopsies.