Inter-rater reliability for diagnosis of geographic atrophy using spectral domain OCT in age-related macular degeneration

Purpose To evaluate the inter-rater reliability for identification of complete retinal pigment epithelium and outer retinal atrophy (cRORA) on SD-OCT images as defined by the Classification of Atrophy Meetings (CAM) group. Methods Fifty images of anonymized SD-OCT line scans of eyes with cRORA due to AMD were selected. Each .tiff image was saved in both black-on-white (BW) and white-on-black (WB) format. Five retina-trained clinicians graded both sets of images twice for the diagnosis of cRORA based on the CAM group definition. Fleiss kappa statistic was calculated for inter-rater reliability and Cohen's kappa statistic for intra-grader and inter-grader reliability between any two graders. Results The inter-grader reliability varied from as low as 0.28 to 0.92 for WB images and 0.34 to 0.86 for BW images. However, the inter-grader and intra-grader agreement was WB 0.92; BW 0.86 and 0.92 respectively, for graders accustomed to the CAM criteria. Fleiss kappa was 0.49 (p value < 0.0001) for WB images and 0.34 (p value < 0.0001 for BW images. Overall, the agreement was better using WB images for all parameters except RPE attenuation/loss. Conclusion There is significant variability in diagnosis of cRORA on SD-OCT by retina-trained ophthalmologists in the real world. The study highlights the need for training to recognise the different features of cRORA prior to its implementation in clinical practice.

Automated summary

The study evaluates the inter-rater reliability for identifying complete retinal pigment epithelium and outer retinal atrophy on SD-OCT images. There is significant variability in diagnosis among retina-trained ophthalmologists, highlighting the need for training to recognize the different features of cRORA before clinical implementation.