Using hierarchical models to analyse clinical indicators: a comparison of the gamma-Poisson and beta-binomial models

Background. Clinical indicators (CIs) are used to assess, compare and determine the potential to improve the care provided by hospitals and physicians. The results for Australian hospitals in 1998-2000 have been reported using a new methodology. The gamma-Poisson hierarchical model was used to correct for the effects of sampling variation by obtaining the empirical Bayesian shrunken estimates for the CI proportions for each hospital. Then, an estimate of the potential system gains that could be achieved if the mean proportion was shifted to the 20th centile is obtained for each of the 185 CIs. The results are used to prioritize quality improvement activity. Objectives. To describe the 20th centile method of calculating potential system gains in the health care system; to determine the impact of using the beta-binomial model rather than the gamma-Poisson model to obtain shrunken estimates for the CI proportions; and to compare the computationally simpler Method of Moments (MoM) with the maximum likelihood (ML) method for parameter estimation. Methods. The formulae for the gamma-Poisson and beta-binomial shrinkage estimators were compared analytically. Each of the shrinkage estimators and the two methods of parameter estimation were applied to the Obstetric and Gynecological CIs, and the results compared. Results. The comparison of the formulae for the two shrinkage estimators showed that the gamma-Poisson model results in greater shrinkage towards the overall mean. This was verified empirically using the clinical indicators. Additionally, the MoM was not a viable alternative to the ML method. Conclusions. The gamma-Poisson model provided smaller estimates of the potential system gains by up to 6.7% of the numerator for the clinical indicators. The difference in estimation increased with increasing mean proportions and between-hospital variation. We recommend that the beta-binomial model should be used on the basis of both theoretical and empirical grounds.