Performance changes due to differences in training data for cerebral aneurysm detection in head MR angiography images

Purpose The performance of computer-aided detection (CAD) software depends on the quality and quantity of the dataset used for machine learning. If the data characteristics in development and practical use are different, the performance of CAD software degrades. In this study, we investigated changes in detection performance due to differences in training data for cerebral aneurysm detection software in head magnetic resonance angiography images. Materials and methods We utilized three types of CAD software for cerebral aneurysm detection in MRA images, which were based on 3D local intensity structure analysis, graph-based features, and convolutional neural network. For each type of CAD software, we compared three types of training pattern, which were two types of training using single-site data and one type of training using multisite data. We also carried out internal and external evaluations. Results In training using single-site data, the performance of CAD software largely and unpredictably fluctuated when the training dataset was changed. Training using multisite data did not show the lowest performance among the three training patterns for any CAD software and dataset. Conclusion The training of cerebral aneurysm detection software using data collected from multiple sites is desirable to ensure the stable performance of the software.

Automated summary

The performance of CAD software depends on the quality and quantity of the dataset used for machine learning. Training using multisite data did not show the lowest performance among the three training patterns for any CAD software and dataset. Using data collected from multiple sites for training is desirable to ensure stable performance of the software.