Plus Disease in Retinopathy of Prematurity A Continuous Spectrum of Vascular Abnormality as a Basis of Diagnostic Variability

被引:72
作者
Campbell, J. Peter [1 ]
Kalpathy-Cramer, Jayashree [2 ]
Erdogmus, Deniz [3 ]
Tian, Peng [3 ]
Kedarisetti, Dharanish [3 ]
Moleta, Chace [1 ]
Reynolds, James D. [4 ]
Hutcheson, Kelly [5 ]
Shapiro, Michael J. [6 ]
Repka, Michael X. [7 ]
Ferrone, Philip [8 ]
Drenser, Kimberly [9 ]
Horowitz, Jason [10 ]
Sonmez, Kemal [11 ]
Swan, Ryan [11 ]
Ostmo, Susan [1 ]
Jonas, Karyn E. [12 ]
Chan, R. V. Paul [12 ]
Chiang, Michael F. [1 ,11 ]
机构
[1] Oregon Hlth & Sci Univ, Dept Ophthalmol, Casey Eye Inst, Portland, OR 97201 USA
[2] Massachusetts Gen Hosp, Dept Radiol, Athinoula A Martinos Ctr Biomed Imaging, Charlestown, MA USA
[3] Northeastern Univ, Cognit Syst Lab, Boston, MA 02115 USA
[4] SUNY Buffalo, Dept Ophthalmol, Ross Eye Inst, Buffalo, NY USA
[5] Sidra Med & Res Ctr, Dept Ophthalmol, Doha, Qatar
[6] Retina Consultants, Chicago, IL USA
[7] Johns Hopkins Univ, Sch Med, Wilmer Inst, Baltimore, MD 21205 USA
[8] Long Isl Vitreoretinal Consultants, Great Neck, NY USA
[9] Oakland Univ, Associated Retinal Consultants, Royal Oak, MI USA
[10] Columbia Univ, Dept Ophthalmol, New York, NY 10027 USA
[11] Oregon Hlth & Sci Univ, Dept Med Informat & Clin Epidemiol, Portland, OR 97201 USA
[12] Univ Illinois, Illinois Eye & Ear Infirm, Dept Ophthalmol & Visual Sci, Chicago, IL USA
基金
美国国家卫生研究院; 美国国家科学基金会;
关键词
ACUTE-PHASE RETINOPATHY; INTEREXPERT AGREEMENT; IMAGE-ANALYSIS; EXPERT; ACCURACY; TELEMEDICINE; TORTUOSITY; FELLOWS; SYSTEM; CARE;
D O I
10.1016/j.ophtha.2016.07.026
中图分类号
R77 [眼科学];
学科分类号
100212 ;
摘要
Purpose: To identify patterns of interexpert discrepancy in plus disease diagnosis in retinopathy of prematurity (ROP). Design: We developed 2 datasets of clinical images as part of the Imaging and Informatics in ROP study and determined a consensus reference standard diagnosis (RSD) for each image based on 3 independent image graders and the clinical examination results. We recruited 8 expert ROP clinicians to classify these images and compared the distribution of classifications between experts and the RSD. Participants: Eight participating experts with more than 10 years of clinical ROP experience and more than 5 peer-reviewed ROP publications who analyzed images obtained during routine ROP screening in neonatal intensive care units. Methods: Expert classification of images of plus disease in ROP. Main Outcome Measures: Interexpert agreement (weighted kappa statistic) and agreement and bias on ordinal classification between experts (analysis of variance [ANOVA]) and the RSD (percent agreement). Results: There was variable interexpert agreement on diagnostic classifications between the 8 experts and the RSD (weighted k, 0-0.75; mean, 0.30). The RSD agreement ranged from 80% to 94% for the dataset of 100 images and from 29% to 79% for the dataset of 34 images. However, when images were ranked in order of disease severity (by average expert classification), the pattern of expert classification revealed a consistent systematic bias for each expert consistent with unique cut points for the diagnosis of plus disease and preplus disease. The 2-way ANOVA model suggested a highly significant effect of both image and user on the average score (dataset A: P < 0.05 and adjusted R-2 = 0.82; and dataset B: P < 0.05 and adjusted R-2 = 0.6615). Conclusions: There is wide variability in the classification of plus disease by ROP experts, which occurs because experts have different cut points for the amounts of vascular abnormality required for presence of plus and preplus disease. This has important implications for research, teaching, and patient care for ROP and suggests that a continuous ROP plus disease severity score may reflect more accurately the behavior of expert ROP clinicians and may better standardize classification in the future. (C) 2016 by the American Academy of Ophthalmology
引用
收藏
页码:2338 / 2344
页数:7
相关论文
共 46 条
[21]   The international classification of retinopathy of prematurity revisited [J].
Gole, GA ;
Ells, AL ;
Katz, X ;
Holmstrom, G ;
Fielder, AR ;
Capone, A ;
Flynn, JT ;
Good, WG ;
Holmes, JM ;
McNamara, JA ;
Palmer, EA ;
Quinn, GE ;
Shapiro, MJ ;
Trese, MGJ ;
Wallace, DK .
ARCHIVES OF OPHTHALMOLOGY, 2005, 123 (07) :991-999
[22]  
Good WV, 2003, ARCH OPHTHALMOL-CHIC, V121, P1684
[23]   Plus Disease in Retinopathy of Prematurity Qualitative Analysis of Diagnostic Process by Experts [J].
Hewing, Nina J. ;
Kaufman, David R. ;
Chan, R. V. Paul ;
Chiang, Michael F. .
JAMA OPHTHALMOLOGY, 2013, 131 (08) :1026-1032
[24]   Measuring agreement in medical informatics reliability studies [J].
Hripcsak, G ;
Heitjan, DF .
JOURNAL OF BIOMEDICAL INFORMATICS, 2002, 35 (02) :99-110
[25]   Plus Disease in Retinopathy of Prematurity Improving Diagnosis by Ranking Disease Severity and Using Quantitative Image Analysis [J].
Kalpathy-Cramer, Jayashree ;
Campbell, J. Peter ;
Erdogmus, Deniz ;
Tian, Peng ;
Kedarisetti, Dharanish ;
Moleta, Chace ;
Reynolds, James D. ;
Hutcheson, Kelly ;
Shapiro, Michael J. ;
Repka, Michael X. ;
Ferrone, Philip ;
Drenser, Kimberly ;
Horowitz, Jason ;
Sonmez, Kemal ;
Swan, Ryan ;
Ostmo, Susan ;
Jonas, Karyn E. ;
Chan, R. V. Paul ;
Chiang, Michael F. .
OPHTHALMOLOGY, 2016, 123 (11) :2345-2351
[26]   PLUS DISEASE DIAGNOSIS IN RETINOPATHY OF PREMATURITY Vascular Tortuosity as a Function of Distance from Optic Disk [J].
Keck, Katie M. ;
Kalpathy-Cramer, Jayashree ;
Ataer-Cansizoglu, Esra ;
You, Sheng ;
Erdogmus, Deniz ;
Chiang, Michael F. .
RETINA-THE JOURNAL OF RETINAL AND VITREOUS DISEASES, 2013, 33 (08) :1700-1707
[27]   Accuracy of retinopathy of prematurity image-based diagnosis by pediatric ophthalmology fellows: Implications for training [J].
Myung, Jane S. ;
Chan, Robison Vernon Paul ;
Espiritu, Michael J. ;
Williams, Steven L. ;
Granet, David B. ;
Lee, Thomas C. ;
Weissgold, David J. ;
Chiang, Michael F. .
JOURNAL OF AAPOS, 2011, 15 (06) :573-578
[28]  
PALMER EA, 1988, ARCH OPHTHALMOL-CHIC, V106, P471
[29]   Validity of a Telemedicine System for the Evaluation of Acute-Phase Retinopathy of Prematurity [J].
Quinn, Graham E. ;
Ying, Gui-Shuang ;
Daniel, Ebenezer ;
Hildebrand, P. Lloyd ;
Ells, Anna ;
Baumritter, Agnieshka ;
Kemper, Alex R. ;
Schron, Eleanor B. ;
Wade, Kelly .
JAMA OPHTHALMOLOGY, 2014, 132 (10) :1178-1184
[30]   Telemedicine Approaches to Evaluating Acute-phase Retinopathy of Prematurity: Study Design [J].
Quinn, Graham E. .
OPHTHALMIC EPIDEMIOLOGY, 2014, 21 (04) :256-267