The fragility of phase 3 trials supporting FDA-approved anticancer medicines: a retrospective analysis

Background The fragility index of trial results-ie, the minimum number of changes from non-events to events resulting in loss of statistical significance-can provide a measure of confidence that a positive effect reported in a randomised controlled trial is real. We aimed to calculate the fragility index of randomised controlled trials supporting US Food and Drug Administration (FDA)-approved anticancer drugs. Methods This is a retrospective analysis of phase 3, randomised, controlled trials supporting anticancer drugs that were approved by the FDA between Jan 1, 2014, and Dec 31, 2018. Two-arm studies with 1:1 randomisation and significant positive results for a time-to-event outcome were eligible for the fragility index calculation, which involves the iterative addition of an event to the experimental group (defined as the group with the smaller number of events in positive trials) and concomitant subtraction of a non-event from that group, until positive significance (defined as p<0.05 by Fisher's exact test) is lost. Findings We identified 36 phase 3 randomised controlled trials, of which 17 (47%) were included in the fragility index analysis. The median fragility index was 2 (IQR 0-27). The fragility index was 2 or less in nine (53%) of 17 trials; for these trials, the fragility index was 1% or less of the total sample size. In five (29%) of 17 trials, the number lost to follow-up was more than the fragility index. Interpretation Many phase 3 randomised controlled trials supporting FDA-approved anticancer drugs have a low fragility index, challenging confidence for concluding their superiority over control treatments. Although not a measure of effect, the fragility index might provide an additional means of assessing the robustness of clinical trial data. Copyright (C) 2019 Elsevier Ltd. All rights reserved.