Forest and funnel plots illustrated the calibration of a prognostic model: a descriptive study

Objective: To describe the use of forest and funnel plots to assess the uniformity of fit in the calibration of a prognostic model for critically ill patients. Methods: This study used the administrative database of the Intensive Care Unit (ICU) at Royal Perth Hospital. This database contained 11,107 consecutive ICU admissions between 1 January 1993 and 31 December 2003. We used forest and funnel plots to assess the calibration of the Acute Physiology and Chronic Health Evaluation (APACHE) II prognostic model. Results: The Mantel-Haenszel weighted standardized mortality ratio (SMR) by pooling different risk or age strata and diagnostic subgroups together was 0.84 (95% Confidence Interval [Cl]: 0.80-0.88), 0.85 (95%CI: 0.80-0.90), and 0.88 (95%CI: 0.82-0.93), respectively. The chi-square for heterogeneity across different risk or age strata and diagnostic subgroups was 10.94 (P = 0.28), 8.92 (P = 0.06), and 24.60 (P < 0.0001), respectively. Funnel plot suggested that the APACHE II model had poor calibration for patients with multiple trauma. The estimates of the slope and intercept of the calibration curve confirmed that the model was poorly calibrated for this subgroup of patients. Conclusion: Forest and funnel plots could facilitate visual interpretation of the calibration of a prognostic model across different subgroups of critically ill patients. (c) 2007 Elsevier Inc. All rights reserved.