Comparison between an empirically derived model and the EuroSCORE system in the evaluation of hospital performance: the example of the Italian CABG outcome project

Objectives: To compare the risk-adjustment model empirically derived from the 'Italian CABG Outcome Project' with that of the additive and logistic EuroSCORE in terms of accuracy, predictive power and ability to rank hospital performances. Methods: The Italian CABG model, the logistic and additive EuroSCORE were applied to the Italian CABG population; the observed deaths/expected deaths (O/E) ratios, as obtained by the three models, were computed for each Italian cardiac surgery centre and for six classes of risk-stratified patients. The performance of the three models in predicting the 30-day mortatity was formally assessed for calibration (Hosmer-Lemeshow test) and discrimination (ROC area). According to the three models, risk-adjusted mortality rates (RAMR = O/E x Italian CABG population mortality rate) were estimated for each centre; possible differences were detected in the identification of hospitals with mortality rates higher and lower than average. Results: The Italian CABG model uses fewer variables than the EuroSCORE system (14 vs 17) and exhibits the best performance in terms of discrimination and calibration. Contrary to the other tested models, the logistic EuroSCORE shows a significant Hosmer-Lemeshow test (chi(2)(H-L) = 19.30, p < 0.0001), indicating unsatisfactory calibration, and a clear predicted death overestimation in each of the considered risk classes (O/E = 0.4). When a proper recalibration procedure is applied, the logistic EuroSCORE performance parameters achieve acceptable levels. The Italian CABG model identified seven centres as having higher than average mortality, while the EuroSCORE identified the same seven centres plus one other. The Italian CABG model identified eight centres with tower than average mortality, five of which were identified by the additive EuroSCORE and four of which were identified by the logistic EuroSCORE. The additive EuroSCORE identified four more and the logistic EuroSCORE three more low mortality centres. Conclusions: Although this analysis reveals a satisfactory concordance between results from the three models, a detailed comparison shows that the Italian CABG model uses fewer variables and performs better than the others. Nevertheless, when property recalibrated, the EuroSCORE model can be exported to the Italian population and used to rank hospital performance and evaluate preoperative risk of patients undergoing open-heart Surgery. (c) 2007 European Association for Cardio-Thoracic Surgery. Published by Elsevier B.V. All rights reserved.