Selective reporting biases in cancer prognostic factor studies

Background. Nonreported and selectively reported information and the use of different definitions may introduce biases in the literature of prognostic factors. We probed these biases in a meta-analysis of a prognostic factor for head and neck squamous cell cancer (HNSCC) mortality that has drawn wide attention-the status of the tumor suppressor protein TP53. Methods: We compared results of meta-analyses that included published data plus unpublished data retrieved from investigators; published data; and only published data indexed with "survival" or "mortality" in MEDLINE/EMBASE, with or without standardized definitions. We also evaluated whether previously published meta-analyses on mortality predictors for various malignancies addressed issues of retrieval and standardized information. All statistical tests were two-sided. Results: For the 18 studies with 1364 patients that included published and indexed data, we obtained a highly statistically significant association between TP53 status and mortality. When we used the definitions preferred by each publication, the association was stronger (risk ratio [RR] = 1.38, 95% confidence interval [CI] = 1.13 to 1.67; P =.001) than when we standardized definitions (RR = 1.27, 95% CI = 1.06 to 1.53; P =.011). The addition of 13 studies with 1028 subjects that included published but not indexed data reduced the observed association (RR = 1.23, 95% CI = 1.03 to 1.47; P =.02). Finally, when we obtained data from investigators (11 studies with 996 patients) and analyzed it with all other data, statistical significance was lost (RR = 1.16, 95% CI = 0.99 to 1.35; P =.06). Among 18 published meta-analyses of 37 cancer prognostic factors, 13 (72%) did not use standardized definitions and 16 (89%) did not retrieve additional information. Conclusions: Selective reporting may spuriously inflate the importance of postulated prognostic factors for various malignancies. We recommend that meta-analyses thereof should maximize retrieval of information and standardize definitions.