4.6 Article

Assessing operating characteristics of CAD algorithms in the absence of a gold standard

期刊

MEDICAL PHYSICS
卷 37, 期 4, 页码 1788-1795

出版社

AMER ASSOC PHYSICISTS MEDICINE AMER INST PHYSICS
DOI: 10.1118/1.3352687

关键词

bootstrapping; diagnostic radiography; lung; maximum likelihood estimation; medical image processing; sampling methods; sensitivity analysis

资金

  1. Science Foundation Ireland [RFP/07/MATF543]
  2. National Institutes of Health [CA109089]

向作者/读者索取更多资源

Methods: A binomial model for multiple reader detections using different diagnostic protocols was constructed, assuming conditional independence of readings given true lesion status. Operating characteristics of all protocols were estimated by maximum likelihood LCA. Reader panel and LCA based estimates were compared using data simulated from the binomial model for a range of operating characteristics. LCA was applied to 36 thin section thoracic computed tomography data sets from the Lung Image Database Consortium (LIDC): Free search markings of four radiologists were compared to markings from four different CAD assisted radiologists. For real data, bootstrap-based resampling methods, which accommodate dependence in reader detections, are proposed to test of hypotheses of differences between detection protocols. Results: In simulation studies, reader panel based sensitivity estimates had an average relative bias (ARB) of -23% to -27%, significantly higher (p-value < 0.0001) than LCA (ARB -2% to -6%). Specificity was well estimated by both reader panel (ARB -0.6% to -0.5%) and LCA (ARB 1.4%-0.5%). Among 1145 lesion candidates LIDC considered, LCA estimated sensitivity of reference readers (55%) was significantly lower (p-value 0.006) than CAD assisted readers' (68%). Average false positives per patient for reference readers (0.95) was not significantly lower (p-value 0.28) than CAD assisted readers' (1.27). Conclusions: Whereas a gold standard based on a consensus of readers may substantially bias sensitivity estimates, LCA may be a significantly more accurate and consistent means for evaluating diagnostic accuracy.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.6
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

Article Computer Science, Artificial Intelligence

Machine-learning-based multiple abnormality prediction with large-scale chest computed tomography volumes

Rachel Lea Draelos, David Dov, Maciej A. Mazurowski, Joseph Y. Lo, Ricardo Henao, Geoffrey D. Rubin, Lawrence Carin

Summary: This study utilized a large-scale chest CT dataset with high-quality abnormality labels, developed a method for extracting labels from radiology reports, and used a deep learning CNN model for multi-organ, multi-disease classification of CT volumes, demonstrating good classification performance.

MEDICAL IMAGE ANALYSIS (2021)

Article Critical Care Medicine

Chest CT Diagnosis and Clinical Management of Drug-Related Pneumonitis in Patients Receiving Molecular Targeting Agents and Immune Checkpoint Inhibitors A Position Paper From the Fleischner Society

Takeshi Johkoh, Kyung Soo Lee, Mizuki Nishino, William D. Travis, Jay H. Ryu, Ho Yun Lee, Christopher J. Ryerson, Tomas Franquet, Alexander A. Bankier, Kevin K. Brown, Jin Mo Goo, Hans-Ulrich Kauczor, David A. Lynch, Andrew G. Nicholson, Luca Richeldi, Cornelia M. Schaefer-Prokop, Johny Verschakelen, Suhail Raoof, Geoffrey D. Rubin, Charles Powell, Yoshikazu Inoue, Hiroto Hatabu

Summary: Diagnosis and management of drug-related pneumonitis requires excluding other potential causes, understanding the incidence and risk factors, and evaluating imaging features based on the distribution of lung parenchymal abnormalities.
Article Respiratory System

Imaging of pulmonary hypertension in adults: a position paper from the Fleischner Society

Martine Remy-Jardin, Christopher J. Ryerson, Mark L. Schiebler, Ann N. C. Leung, James M. Wild, Marius M. Hoeper, Philip O. Alderson, Lawrence R. Goodman, John Mayo, Linda B. Haramati, Yoshiharu Ohno, Patricia Thistlethwaite, Edwin J. R. van Beek, Shandra Lee Knight, David A. Lynch, Geoffrey D. Rubin, Marc Humbert

Summary: Pulmonary hypertension (PH) is characterized by a mean pulmonary artery pressure greater than 20 mmHg and classified into five groups with similar pathophysiologic mechanisms. Radiologists play a key role in the multidisciplinary assessment and management of PH, utilizing CT, MRI, and nuclear medicine to evaluate and treat the condition effectively.

EUROPEAN RESPIRATORY JOURNAL (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

Imaging of Pulmonary Hypertension in Adults: A Position Paper from the Fleischner Society

Martine Remy-Jardin, Christopher J. Ryerson, Mark L. Schiebler, Ann N. C. Leung, James M. Wild, Marius M. Hoeper, Philip O. Alderson, Lawrence R. Goodman, John Mayo, Linda B. Haramati, Yoshiharu Ohno, Patricia Thistlethwaite, Edwin J. R. van Beek, Shandra Lee Knight, David A. Lynch, Geoffrey D. Rubin, Marc Humbert

Summary: Pulmonary hypertension is a condition characterized by elevated mean pulmonary artery pressure, with radiologists playing a crucial role in its assessment and management. A working group was established to focus on the role of imaging techniques in diagnosing and managing PH, highlighting the importance of imaging in the recognition, work-up, treatment planning, and follow-up of PH.

RADIOLOGY (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

Chest CT Diagnosis and Clinical Management of Drug-related Pneumonitis in Patients Receiving Molecular Targeting Agents and Immune Checkpoint Inhibitors: A Position Paper from the Fleischner Society

Takeshi Johkoh, Kyung Soo Lee, Mizuki Nishino, William D. Travis, Jay H. Ryu, Ho Yun Lee, Christopher J. Ryerson, Tomas Franquet, Alexander A. Bankier, Kevin K. Brown, Jin Mo Goo, Hans-Ulrich Kauczor, David A. Lynch, Andrew G. Nicholson, Luca Richeldi, Cornelia M. Schaefer-Prokop, Johny Verschakelen, Suhail Raoof, Geoffrey D. Rubin, Charles Powell, Yoshikazu Inoue, Hiroto Hatabu

Summary: The frequency and broad spectrum of lung toxicity have increased with the use of molecular targeting agents and immune checkpoint inhibitors (ICIs), particularly in cancer patients. Diagnosis of drug-related pneumonitis (DRP) involves excluding other causes, and awareness of its incidence and risk factors is crucial. The severity of DRP symptoms can range from mild to life-threatening, requiring accurate diagnosis and prompt treatment.

RADIOLOGY (2021)

Review Radiology, Nuclear Medicine & Medical Imaging

COVID-19 Imaging: What We Know Now and What Remains Unknown

Jeffrey P. Kanne, Harrison Bai, Adam Bernheim, Michael Chung, Linda B. Haramati, David F. Kallmes, Brent P. Little, Geoffrey D. Rubin, Nicola Sverzellati

Summary: The role of imaging, specifically CT scans, evolved during the pandemic, initially being seen as an alternative and potentially superior testing method compared to RT-PCR, but later having a more limited role based on specific indications.

RADIOLOGY (2021)

Article Cardiac & Cardiovascular Systems

The direct costs of coronary CT angiography relative to contrast-enhanced thoracic CT: Time-driven activity-based costing

Michael E. Zimmerman, Juan C. Batlle, Cathleen Biga, Ron Blankstein, Brian B. Ghoshhajra, Mark G. Rabbat, George E. Wesbey, Geoffrey D. Rubin

Summary: The study found that the direct cost of performing Coronary CT angiography (CCTA) is significantly higher than Contrast-enhanced thoracic CT (CECT), with both labor and equipment costs being more expensive for CCTA. This suggests that reimbursement schedules treating these procedures similarly undervalue the resources required for CCTA, potentially limiting access to this procedure.

JOURNAL OF CARDIOVASCULAR COMPUTED TOMOGRAPHY (2021)

Article Medical Informatics

Multi-label annotation of text reports from computed tomography of the chest, abdomen, and pelvis using deep learning

Vincent M. D'Anniballe, Fakrul Islam Tushar, Khrystyna Faryna, Songyue Han, Maciej A. Mazurowski, Geoffrey D. Rubin, Joseph Y. Lo

Summary: This study developed a high-throughput multi-label annotator for body CT reports using a dictionary approach and rule-based algorithms for disease label extraction, with attention-guided recurrent neural networks for classification. The method showed excellent accuracy and performance, able to adapt to various cases and diseases, suitable for automated labeling of large-scale medical datasets.

BMC MEDICAL INFORMATICS AND DECISION MAKING (2022)

Article Cardiac & Cardiovascular Systems

CAD-RADSTM 2.0-2022 Coronary Artery Disease-Reporting and Data System An Expert Consensus Document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Cardiology (ACC), the American College of Radiology (ACR), and the North America Society of Cardiovascular Imaging (NASCI)

Ricardo C. Cury, Jonathon Leipsic, Suhny Abbara, Stephan Achenbach, Daniel Berman, Marcio Bittencourt, Matthew Budoff, Kavitha Chinnaiyan, Andrew D. Choi, Brian Ghoshhajra, Jill Jacobs, Lynne Koweek, John Lesser, Christopher Maroules, Geoffrey D. Rubin, Frank J. Rybicki, Leslee J. Shaw, Michelle C. Williams, Eric Williamson, Charles S. White, Todd C. Villines, Ron Blankstein

Summary: Coronary Artery Disease Reporting and Data System (CAD-RADS) is a standardized reporting system for patients undergoing coronary CT angiography (CCTA), aiming to guide patient management. The updated CAD-RADS 2.0 improves the initial reporting system for CCTA by considering new technical developments and clinical guidelines, including the assessment of stenosis, plaque burden, and modifiers.

JOURNAL OF CARDIOVASCULAR COMPUTED TOMOGRAPHY (2022)

Article Cardiac & Cardiovascular Systems

CAD-RADSTM 2.0-2022 Coronary Artery Disease-Reporting and Data System

Ricardo C. Cury, Jonathon Leipsic, Suhny Abbara, Stephan Achenbach, Daniel Berman, Marcio Bittencourt, Matthew Budoff, Kavitha Chinnaiyan, Andrew D. Choi, Brian Ghoshhajra, Jill Jacobs, Lynne Koweek, John Lesser, Christopher Maroules, Geoffrey D. Rubin, Frank J. Rybicki, Leslee J. Shaw, Michelle C. Williams, Eric Williamson, Charles S. White, Todd C. Villines, Ron Blankstein

Summary: The Coronary Artery Disease Reporting and Data System (CAD-RADS) was created to standardize reporting system for patients undergoing coronary CT angiography (CCTA) and guide patient management. The updated 2022 CAD-RADS 2.0 aims to improve the initial reporting system for CCTA by considering new technical developments in cardiac CT, including data from recent clinical trials and new clinical guidelines.

JACC-CARDIOVASCULAR IMAGING (2022)

Article Radiology, Nuclear Medicine & Medical Imaging

CAD-RADS™ 2.0-2022 Coronary Artery Disease - Reporting and Data System. An expert consensus document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Cardiology (ACC), the American College of Radiology (ACR) and the North America Society of Cardiovascular Imaging (NASCI)

Ricardo C. Cury, Jonathon A. Leipsic, Suhny Abbara, Stephan Achenbach, Daniel S. Berman, Marcio Bittencourt, Matthew Budoff, Kavitha Chinnaiyan, Andrew D. Choi, Brian Ghoshhajra, Jill Jacobs, Lynne Koweek, John Lesser, Christopher Maroules, Geoffrey D. Rubin, Frank J. Rybicki, Leslee J. Shaw, Michelle C. Williams, Eric Williamson, Charles S. White, Todd C. Villines, Ron Blankstein

Summary: Coronary Artery Disease Reporting and Data System (CAD-RADS) is a standardized reporting system for CCTA patients, aiming to guide patient management. The updated CAD-RADS 2.0 improves the initial reporting system by considering new technical developments, clinical trial data, and clinical guidelines.

JOURNAL OF THE AMERICAN COLLEGE OF RADIOLOGY (2022)

Article Radiology, Nuclear Medicine & Medical Imaging

CAD-RADSTM 2.0 - 2022 Coronary Artery Disease - Reporting and Data System An Expert Consensus Document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Cardiology (ACC), the American College of Radiology (ACR) and the North America Society of Cardiovascular Imaging (NASCI)

Ricardo C. Cury, Jonathon Leipsic, Suhny Abbara, Stephan Achenbach, Daniel Berman, Marcio Bittencourt, Matthew Budoff, Kavitha Chinnaiyan, Andrew D. Choi, Brian Ghoshhajra, Jill Jacobs, Lynne Koweek, John Lesser, Christopher Maroules, Geoffrey D. Rubin, Frank J. Rybicki, Leslee J. Shaw, Michelle C. Williams, Eric Williamson, Charles S. White, Todd C. Villines, Ron Blankstein

Summary: The Coronary Artery Disease Reporting and Data System (CAD-RADS) is a standardized reporting system for patients who undergo coronary CT angiography (CCTA) and aims to guide the management of patients. The updated 2022 CAD-RADS 2.0 improves the initial reporting system for CCTA by considering new technical developments, clinical trials, and guidelines. The classification in CAD-RADS follows a framework of stenosis, plaque burden, and modifiers, with the addition of assessing lesion-specific ischemia. It provides a standardized framework for communication, education, research, and improving patient care.

RADIOLOGY-CARDIOTHORACIC IMAGING (2022)

Proceedings Paper Engineering, Biomedical

Co-occurring Diseases Heavily Influence the Performance of Weakly Supervised Learning Models for Classification of Chest CT

Fakrul Islam Tushar, Vincent M. D'Anniballe, Geoffrey D. Rubin, Ehsan Samei, Joseph Y. Lo

Summary: This paper examines the effect of co-occurring diseases on weakly supervised learning in computer-aided diagnosis. The results show that binary classifiers outperform multi-label classifiers in every disease category. However, the performance of binary classifiers is heavily influenced by co-occurring diseases, indicating that they may not always learn the correct appearance of specific diseases.

MEDICAL IMAGING 2022: COMPUTER-AIDED DIAGNOSIS (2022)

Article Computer Science, Artificial Intelligence

Classification of Multiple Diseases on Body CT Scans Using Weakly Supervised Deep Learning

Fakrul Islam Tushar, Vincent M. D'Anniballe, Rui Hou, Maciej A. Mazurowski, Wanyi Fu, Ehsan Samei, Geoffrey D. Rubin, Joseph Y. Lo

Summary: This study aims to design multidisease classifiers for body CT scans for three different organ systems using automatically extracted labels from radiology text reports. The results of the study demonstrate that weakly supervised deep learning models can effectively classify different diseases in multiple organ systems and achieve relatively high accuracy.

RADIOLOGY-ARTIFICIAL INTELLIGENCE (2022)

Article Radiology, Nuclear Medicine & Medical Imaging

Task-dependent estimability index to assess the quality of cardiac computed tomography angiography for quantifying coronary stenosis

Ehsan Samei, Taylor Richards, William P. Segars, Melissa A. Daubert, Alex Ivanov, Geoffrey D. Rubin, Pamela S. Douglas, Udo Hoffmann

Summary: A computational framework was developed to objectively assess the precision of quantifying coronary stenosis in cardiac CTA, showing promising results in predicting image quality and estimation precision. This framework was successfully applied in a clinical trial and demonstrated potential for optimizing imaging protocols for targeted precision and measurement consistency in cardiac CT images.

JOURNAL OF MEDICAL IMAGING (2021)

暂无数据