Natural language processing to identify ureteric stones in radiology reports

Introduction Natural language processing (NLP) is an emerging tool which has the ability to automate data extraction from large volumes of unstructured text. One of the main described uses of NLP in radiology is cohort building for epidemiological studies. This study aims to assess the accuracy of NLP in identifying a group of patients positive for ureteric stones on Computed Tomography - Kidneys, Ureter, Bladder (CT KUB) reports. Methods Retrospective review of all CT KUB reports in a single calendar year. A locally available NLP tool was used to automatically classify the reports based on positivity for ureteric stones. This was validated by manual review and refined to maximize the accuracy of stone detection. Results A total of 1874 CT KUB reports were identified. Manual classification of ureteric stone positivity was 36% compared with 27% using NLP. The accuracy of NLP was 85% with a sensitivity of 66% and specificity of 95%. Incorrect classification was due to misspellings, variable syntax, terminology, pluralization and the inability to exclude clinical request details from the search algorithm. Conclusions Our NLP tool demonstrated high specificity but low sensitivity at identifying CT KUB reports that are positive for ureteric stones. This was attributable to the lack of feature extraction tools tailored for analysing radiology text, incompleteness of the medical lexicon database and heterogeneity of unstructured reports. Improvements in these areas will help improve data extraction accuracy.