Performance of a Deep-Learning Algorithm vs Manual Grading for Detecting Diabetic Retinopathy in India

Key PointsQuestionWhat is the performance of a deep-learning model in a cohort of patients with diabetes in India? FindingsIn this observational study of moderate or worse diabetic retinopathy and referable diabetic macular edema, the automated diabetic retinopathy system's performance was equal to or exceeded manual grading. MeaningDeep-learning models performed well within a population of patients with diabetes from India. ImportanceMore than 60 million people in India have diabetes and are at risk for diabetic retinopathy (DR), a vision-threatening disease. Automated interpretation of retinal fundus photographs can help support and scale a robust screening program to detect DR. ObjectiveTo prospectively validate the performance of an automated DR system across 2 sites in India. Design, Setting, and ParticipantsThis prospective observational study was conducted at 2 eye care centers in India (Aravind Eye Hospital and Sankara Nethralaya) and included 3049 patients with diabetes. Data collection and patient enrollment took place between April 2016 and July 2016 at Aravind and May 2016 and April 2017 at Sankara Nethralaya. The model was trained and fixed in March 2016. InterventionsAutomated DR grading system compared with manual grading by 1 trained grader and 1 retina specialist from each site. Adjudication by a panel of 3 retinal specialists served as the reference standard in the cases of disagreement. Main Outcomes and MeasuresSensitivity and specificity for moderate or worse DR or referable diabetic macula edema. ResultsOf 3049 patients, 1091 (35.8%) were women and the mean (SD) age for patients at Aravind and Sankara Nethralaya was 56.6 (9.0) years and 56.0 (10.0) years, respectively. For moderate or worse DR, the sensitivity and specificity for manual grading by individual nonadjudicator graders ranged from 73.4% to 89.8% and from 83.5% to 98.7%, respectively. The automated DR system's performance was equal to or exceeded manual grading, with an 88.9% sensitivity (95% CI, 85.8-91.5), 92.2% specificity (95% CI, 90.3-93.8), and an area under the curve of 0.963 on the data set from Aravind Eye Hospital and 92.1% sensitivity (95% CI, 90.1-93.8), 95.2% specificity (95% CI, 94.2-96.1), and an area under the curve of 0.980 on the data set from Sankara Nethralaya. Conclusions and RelevanceThis study shows that the automated DR system generalizes to this population of Indian patients in a prospective setting and demonstrates the feasibility of using an automated DR grading system to expand screening programs. This study assesses the validity of an automated diabetic retinopathy system compared with manual grading across 2 sites in India.