Does observer bias contribute to variations in the rate of retinopathy of prematurity between centres?

Purpose: We aimed to indirectly assess the contribution from observer bias to between centre variability in the incidence of acute retinopathy of prematurity (ROP). Methods: The Australian and New Zealand Neonatal Network (ANZNN) collected data on the highest stage of acute ROP in either eye in 2286 infants born at less than 29 weeks in 1998-1999 and cared for in one of 25 neonatal intensive care units (NICUs). Chi-squared analysis was used to detect differences in the proportion of stages of ROP for each neonatal intensive care unit. These proportions were compared with those reported in two large studies of treatment for ROP. Results: The incidence of acute ROP in the ANZNN cohort was 42% and the ratio of stage 1:2:3 ROP was 1.5:1.9:1. There was considerable variation in both the incidence of acute ROP and the proportions with stage 1:2:3 ROP between centres. A chi-squared test determined that the assignment of stages 1, 2 and 3/4 ROP was not independent of centre (chi(2)(48) = 165.2; P < 0.0001). Treatment of stage 3 ROP varied between 15% and 120%, indicating some eyes were treated at less than stage 3. Conclusion: The data are highly suggestive of observer bias contributing to the observed between centre variation in the incidence of acute ROP. In neonatal intervention studies where acute ROP is an outcome it would seem important to have an accreditation process for examining ophthalmologists, and there are similar arguments for neonatal networks which collect these data.