Comparing results of large clinical trials to those of meta-analyses

We consider methods for assessing agreement or disagreement between the results of a meta-analysis of small studies addressing a clinical question and the result of a large clinical trial (LCT) addressing the same clinical question. We recommend basing conclusions about agreement upon the difference between the two results (relative risk, log-odds ratio or similar summary statistic), in the light of the estimated standard error of that difference. To estimate the standard error of the meta-analytic result we recommend a random effects analysis, and where a between-studies variance component is found, that component of variance should be used twice: once in the estimated standard error for the meta-analytic result and again in the standard error of the LCT result (augmenting the internal standard error of that statistic). Such broadening of the standard error reduces the appearance of disagreement. We also offer a critique of a different published approach, which is based on consistency of findings of statistical significance, a matter of how the two results regard zero, which is a poor measure of how closely they agree with each other. Copyright (C) 2002 John Wiley Sons, Ltd.