Apples and Oranges: Comparison of ACS-NSQIP Observed Outcomes With Premier's Quality Manager-Predicted Outcomes

The National Surgical Quality Improvement Program (NSQIP) is used by the American College of Surgeons to measure and report surgical quality and outcomes. Premier's Quality Manager (QM) generates expected outcomes from patient charts. The authors compared observed NSQIP morbidity and mortality outcomes with those predicted by QM. NSQIP data for 1919 patients were entered into QM. The discriminatory accuracy of the QM model was assessed using the C statistic (1.0 implies perfect discrimination, and 0.5 implies no discrimination). NSQIP and QM both identified 51 deaths (C statistic, 0.91). NSQIP identified 478 postoperative occurrences, whereas QM predicted 714 patients with at least 1 complication; 223 of these were subclassified as patients with at least 1 morbid complication (C statistic, 0.83). QM did not perform as well in predicting the observed NSQIP morbidities. Surgical leaders and hospital administrators must critically evaluate products before adopting programs designed to improve patient outcomes or making decisions regarding physician practice.