The science of systematic reviewing studies of diagnostic tests

Background: Systematic reviews have gradually replaced single studies as the highest level of documented effectiveness of health care interventions. Systematic reviewing is a new scientific method, concerned with the development and application of methods for identifying relevant literature, analysing the material while increasing validity and precision, and presenting and discussing the results in a way that does justice to the research question and to the available evidence. The objective of this study was to review the systematic reviews in laboratory medicine, to evaluate the methods applied in these reviews and the applicability of guidelines of the Cochrane Methods Working Group on Screening and Diagnostic Tests, and identify areas for future research. Methods: All the systematic reviews in the field of clinical chemistry and laboratory haematology that could be identified in Medline, EMBASE and other literature databases up to December 1998, were evaluated. Results: We studied 23 reviews of diagnostic trials. Although all reviews share the same basic methodology, there was a wide variation in the methods applied. There was no consensus on the quality criteria for inclusion of primary studies. The results of the primary studies were heterogeneous in most cases. This was partly due to design flaws in the primary studies, but was also inherent in the diverse study designs in diagnostic trials. We observed differences in the analysis of the factors that cause heterogeneity of the results, and in the summary statistics used to pool the data from the primary studies. The additional diagnostic value of a test, after other test results are taken into consideration, was only addressed in one study. Conclusion: This overview of 23 reviews of diagnostic trials identifies areas in the methods of systematic reviewing where consensus is lacking, such as quality rating of primary studies, analysis of heterogeneity between primary studies and pooling of data. Guidelines need to be improved on these points.