Quality of systematic reviews used in guidelines for oncology practice

Background: Systematic reviews are an important tool for developing clinical recommendations. Those of high quality assure a good level of confidence on the strength of the recommendations. Methods: A QUOROM-based checklist was applied to the reviews cited in a sample of guidelines on breast and colon cancer prevention and therapy. The checklist provided a weight for each criterion and a total quality score. Each review was independently evaluated by two reviewers; disagreements were solved by consensus. Results: Eighty reviews (96%) were retrieved and evaluated; 36 focused on breast, and 44 on colorectal cancer. Twenty-three reviews (29%) did not match the definition of systematic review. In 17 (21%) the searching methods were unclear or described elsewhere. Forty (50%) were systematic. Not systematic, low and very low quality reviews accounted for 70% of the total. No review obtained the A+ class score; only 5 (6%) the A- and 7 (9%) the B+. Conclusions: The results of this assessment provide a sober picture of the quality of the sources used to build guidelines. Oncologists should be aware that they could be relying on poor underlying documents. Writing groups should be aware of methodological problems, and should consult the existing manuals for the preparation of guidelines.