Remote image based retinopathy of prematurity diagnosis: a receiver operating characteristic analysis of accuracy

Background/aims: Telemedicine offers potential to improve the accessibility and quality of diagnosis of retinopathy of prematurity ( ROP). The aim of this study was to measure accuracy of remote image based ROP diagnosis by three readers using receiver operating characteristic ( ROC) analysis. Methods: 64 hospitalised infants who met ROP examination criteria underwent two consecutive bedside procedures: dilated examination by an experienced paediatric ophthalmologist and digital retinal imaging with a commercially available wide angle camera. 410 images from 163 eyes were reviewed independently by three trained ophthalmologist readers, who classified each eye into one of four categories: no ROP, mild ROP, type 2 prethreshold ROP, or ROP requiring treatment. Sensitivity and specificity for detection of mild or worse ROP, type 2 prethreshold or worse ROP, and ROP requiring treatment were determined, compared to a reference standard of dilated ophthalmoscopy. ROC curves were generated by calculating values for each reader at three diagnostic cut-off levels: mild or worse ROP ( that is, reader was asked whether image sets represented mild or worse ROP), type 2 prethreshold or worse ROP ( that is, reader was asked whether image sets represented type 2 prethreshold or worse ROP), and ROP requiring treatment. Results: Areas under ROC curves ranged from 0.747 - 0.896 for detection of mild or worse ROP, 0.905 - 0.946 for detection of type 2 prethreshold or worse ROP, and 0.941 - 0.968 for detection of ROP requiring treatment. Conclusions: Remote interpretation is highly accurate among multiple readers for the detection of ROP requiring treatment, but less so for detection of mild or worse ROP.