SHOULD UNPUBLISHED DATA BE INCLUDED IN METAANALYSES - CURRENT CONVICTIONS AND CONTROVERSIES

Objective.-To identify the extent to which meta-analyses currently include unpublished data and whether editors, meta-analysts, and methodologists believe unpublished material should be included. Design.-This article describes two related studies: a literature review and a cross-sectional survey. Sample Selection.-For the literature review, we identified all articles indexed by the key word meta-analysis from January 1989 to February 1991 and determined whether unpublished material had been searched for, obtained, and included in the meta-analyses. For the cross-sectional survey, we surveyed authors of these meta-analyses, authors of articles addressing methodological issues in meta-analysis published during the same period, and editors of journals in which both types of articles were published. Intervention.-We asked the respondents about their attitudes concerning inclusion of unpublished data and publication of articles from which data had previously been included in a scientific overview. Main Outcome Measures.-Inclusion of unpublished data and opinions about whether unpublished material should be included in overviews and whether prior inclusion of data in an overview should bear on publication. Results.-Of 150 meta-analyses, 46 (30.7%) included unpublished data in their primary analysis. Of authors surveyed, 85% responded. Of the meta-analysts and methodologists, 77.7% felt that unpublished material should definitely or probably be included in scientific overviews; this was true of 46.9% of the editors. A total of 86.4% of the meta-analysts and methodologists and 53.2% of the editors felt that inclusion of data in a prior overview should have no bearing on full publication of original research. Conclusion.-While inclusion of unpublished data in scientific overviews remains controversial, most investigators directly involved in meta-analysis believe that unpublished data should not be systematically excluded. The most valid synthesis of available information will result when meta-analysts subject published and unpublished material to the same rigorous methodological evaluation and present results with and without unpublished sources of data.