High prevalence but low impact of data extraction and reporting errors were found in Cochrane systematic reviews

Background and Objective: Extracting data from primary articles is an essential component in conducting systematic reviews. Incorrect data extraction can lead to false conclusions. The objective of this study was to retrospectively repeat the data extraction in all systematic reviews conducted by the Cochrane Cystic Fibrosis and Genetic Disorders Group. Study Design and Setting: For each review, data extraction was conducted, by an experienced statistician, for the same publications used by the reviewers. Results were compared with those obtained by the reviewers. Results: Errors were found in 20 of 34 reviews, including incorrect calculations made when converting data in primary articles into data required for the review (2 reviews) and misinterpretation of data that were reported in the primary article (7 reviews). All data-handling errors led to changes in the summary results, but none of these affected the review conclusions. Conclusions: Important errors were identified in a high proportion of reviews. A variety of problems relating to the reporting of results within a review were identified, but these did not lead to substantial changes in any conclusion. (c) 2005 Elsevier Inc. All rights reserved.