A comparison of neonatal mortality risk prediction models in very low birth weight infants

Background. Risk-adjusted severity of illness is frequently used in clinical research and quality assessments. Although there are multiple methods designed for neonates, they have been infrequently compared and some have not been assessed in large samples of very low birth weight (VLBW; <1500 g) infants. Objectives. To test and compare published neonatal mortality prediction models, including Clinical Risk Index for Babies (CRIB), Score for Neonatal Acute Physiology (SNAP), SNAP-Perinatal Extension (SNAP-PE), Neonatal Therapeutic Interventions Scoring System, the National Institute of Child Health and Human Development (NICHD) network model, and other individual admission factors such as birth weight, low Apgar score (< 7 at 5 minutes), and small for gestational age status in a cohort of VLBW infants from the Washington, DC area. Methods. Data were collected on 476 VLBW infants admitted to 8 neonatal intensive care units between October 1994 and February 1997. The calibration (closeness of total observed deaths to the predicted total) of models with published coefficients (SNAP-PE, CRIB, and NICHD) was assessed using the standardized mortality ratio. Discrimination was quantified as the area under the curve (AUC) for the receiver operating characteristic curves. Calibrated models were derived for the current database using logistic regression techniques. Goodness-of-fit of predicted to observed probabilities of death was assessed with the Hosmer-Lemeshow goodness-of-fit test. Results. The calibration of published algorithms applied to our data was poor. The standardized mortality ratios for the NICHD, CRIB, and SNAP-PE models were .65, .56, and .82, respectively. Discrimination of all the models was excellent (range: .863-.930). Surprisingly, birth weight performed much better than in previous analyses, with an AUC of .869. The best models using both 12- and 24-hour postadmission data, significantly outperformed the best model based on birth data only but were not significantly different from each other. The variables in the best model were birth weight, birth weight squared, low 5-minute Apgar score, and SNAP (AUC = .930). Conclusion. Published models for severity of illness overpredicted hospital mortality in this set of VLBW infants, indicating a need for frequent recalibration. Discrimination for these severity of illness scores remains excellent. Birth variables should be reevaluated as a method to control for severity of illness in predicting mortality.