Meta-analysis of binary data: which within study variance estimate to use?

We applied a mixed effects model to investigate between- and within-study variation in improvement rates of 180 schizophrenia outcome studies. The between-study variation was explained by the fixed study characteristics and an additional random study effect. Both rate difference and logit models were used. For a binary proportion outcome p(i) with sample size n(i) in the ith study, ((p) over cap (i)(1 - (p) over cap (i))n(i))(-1) is the usual estimate of the within-study variance sigma (2)(i) in the logit model, where (p) over cap (i) is the sample mean of the binary outcome for subjects in study i. This estimate can be highly correlated with logit((p) over cap (i)). we used ((p) over bar (i)(1 - (p) over bar )n(i))(-1) as an alternative estimate of sigma (2)(i), where (p) over bar is the weighted mean of (p) over cap (i)'s. We estimated regression coefficients (beta) of the fixed effects and the variance (tau (2)) of the random study effect using a quasi-likelihood estimating equations approach. Using the schizophrenia meta-analysis data, we demonstrated how the choice of the estimate of sigma (2)(i) affects the resulting estimates of beta and tau (2). We also conducted a simulation study to evaluate the performance of the two estimates of sigma (2)(i) in different conditions, where the conditions vary by number of studies and study size. Using the schizophrenia meta-analysis data, the estimates of beta and tau (2) were quite different when different estimates of sigma (2)(i) were used in the logit model. The simulation study showed that the estimates of beta and tau (2) were less biased, and the 95 per cent CI coverage was closer to 95 per cent when the estimate of sigma (2)(i) was ((p) over bar (1 - (p) over bar )n(i))(-1) rather than ((p) over cap (i)(1 - (p) over cap (i))n(i))(-1). Finally, we showed that a simple regression analysis is not appropriate unless tau (2) is much larger than sigma (2)(i), or a robust variance is used. Copyright (C) 2001 John Wiley & Sons, Ltd.