Are randomized clinical trials good for us (in the short term)? Evidence for a "trial effect"

Objective: To assess whether there is evidence that randomized controlled trials are systematically beneficial, or harmful, for patients. In other words, is there a "trial effect"? If so, to examine whether the evidence sheds light on the likely sources of the difference in outcomes. Methods: Systematic review of the literature. Results: We set out in some detail potential sources of a "trial effect" and potential biases. We found only 14 research articles (covering more than 21 trials) with relevant primary data. We extracted, with difficulty, quantitative data-sets from the articles, and classified these according to likely source of any apparent trial effect. The categories used were: differences in prognosis; superior treatment in the trial; and "protocol/Hawthorne effect" (benefit from improved routine care within a trial). Analysis: The evidence available is limited in breadth (coming largely from cancer trials) and quality, as well as quantity. There is weak evidence to suggest that clinical trials have a positive effect on the outcome of participants. This does not appear to depend strongly on the trial demonstrating that an experimental treatment is superior. However, benefit to participants is less evident where scope for a "protocol/Hawthorne effect" was apparently limited (because there was no effective routine treatment or because the comparison group also received protocol care). A form of bias, arising if clinicians who tend to recruit to trials also tend to be better clinicians, could also explain these results. Conclusion: While the evidence is not conclusive, it is more likely that clinical trials have a positive rather than a negative effect on the outcome of patients. In the limited data available, the effect seems to be larger in trials where an effective treatment already exists and is included in the trial protocol. Recommendation: That carefully researched treatment protocols, and monitoring of outcomes, be used for all patients, not just those in trials. (C) 2001 Elsevier Science Inc. All rights reserved.