Comparing outcomes of coronary artery bypass surgery: Is the New York Cardiac Surgery Reporting System model sensitive to changes in case mix?

Objective: To assess the validity of using the standardized mortality ratio (SMR), based on the New York State Cardiac Surgery Reporting System (CSRS) prediction model to compare coronary artery bypass grafting (CABG) outcomes between hospitals. Design: The study was designed as a retrospective study based on a database containing all patients undergoing isolated CABG surgery in New York State hospitals in 1996 (n = 20,078). In the first part of this study, a computer simulation was used to assess the impact of case mix variation on the SMR. A computer-intensive algorithm was used to create 5,000 random case mixes from the patients in the CSRS database. The SMR associated with each of the 5,000 case mixes was calculated using a resampling algorithm. The second part of this study was designed to determine whether the identity of quality outliers among all of the 32 hospitals in the CSRS database would change after adjusting for the effects of case mix on the SMR. The SMR associated with the case mix of each hospital in the CSRS database (the hospital case mix SMR) was obtained using a resampling algorithm. The hospital SMR (as well as 95% confidence interval) was then calculated using bootstrapping for each of the 32 hospitals within the CSRS database. An adjusted SMR was then derived for each hospital by dividing the hospital SMR by the case mix SMR for that hospital. Setting. Thirty-two hospitals in New York State performing CABG surgery. Interventions., None. Results: Changes in patient case mix are associated with statistically significant changes in the SMR. However, there was no difference in the identity of quality outliers in the New York State CSRS database when using either the SMR or the SMR adjusted for the effects of case mix. Conclusion: Risk-adjusted measures of outcomes in CABG patients may be potentially biased by differences in case mix between institutions because of the influence of case mix on the process of risk adjustment. There was, however, no evidence of bias in the specific application of the CSRS model to the hospitals in the CSRS database.