Application of credibility ceilings probes the robustness of meta-analyses of biomarkers and cancer risk

Objectives: Meta-analyses of biomarkers often present spurious significant results and large effects. We applied sensitivity analyses with the use of credibility ceilings to assess whether and how the results of meta-analyses of biomarkers and cancer risk would change. Study Design and Setting: We evaluated 98 meta-analyses, 43 (44%) of which had nominally statistically significant results. We assumed that any single study cannot give more than a maximum certainty 100 - c% (c, credibility ceiling) that the effect estimate [odds ratio (OR)] exceeds 1 (null) or 1.2. Results: Nominal statistical significance was maintained for 21(21%) meta-analyses, for c = 10% and OR > 1, and these proportions changed to 7%, 3%, and 6% with ceilings of 20%, 30%, and 40%, respectively. For ceilings for OR > 1.2, the respective proportions were 37%, 21%, 7%, and 3%. Seven meta-analyses on infectious agents retained statistical significance even with a high ceiling of c = 20% for OR > 1.00. Meta-analyses without other hints of bias (large between-study heterogeneity, small-study effects, excess significance) were more likely to retain statistical significance than those that had such hints of bias. Conclusion: Credibility ceilings may be helpful in meta-analyses of biomarkers to understand the robustness of the results to different levels of uncertainty. (C) 2015 Elsevier Inc. All rights reserved.