Bias in the design and reporting of oncology trials
Bias can enter an oncology trial at its design and carry through to its reporting. Each confirmatory design carries its own conditions, each design weakness distorts the effect estimate in its own way, and a confirmatory trial needs an estimand framework. Active reporting bias is the kind a writer works to avoid, while responsibility for the design is shared.
Confirmatory designs and their conditions
Choosing the study design and the endpoints or estimands needs thorough and careful consideration at the planning stage of an oncology trial, because any bias introduced at the outset of a trial cascades down to the analysis and eventually to the reporting of the results.
Confirmatory evidence has to come from adequate and well-controlled trials: a valid control, appropriate eligibility criteria, robust randomisation, proper masking of participants, observers and analysts, well-defined and reliable endpoints, and sound analysis. A randomised, controlled, blinded superiority trial on clinically meaningful endpoints, against placebo, an active comparator or both, is arguably the standard and is considered the most reliable design to demonstrate efficacy. When superiority is shown, its data are relatively easy to interpret. Each alternative carries its own conditions. Where an active control is compared with the investigational drug, a noninferiority design can show that the difference between them stays within a prespecified minimum margin. Its important considerations, often also its challenges, are an active control with a well-characterised effect, a control effect size and margin taken from reliable historical data, a sample size powered on the expected effects of both drugs, and a carefully chosen hypothesis and test. A single-arm design may be acceptable where no active therapy exists and placebo would be unethical, or where the patient pool is limited, as in rare diseases. It has to be justified, because its inherent drawbacks may limit the validity and generalisability of the efficacy data: adverse-event causality is hard to assess, there is no comparison to ascertain the real effect, and sample size and endpoints are restricted. Crossover in oncology typically moves patients on placebo whose disease progresses onto an arm that shows benefit, for ethical and accrual reasons. It is considered riskier than in symptomatic chronic disease or single-dose pharmacology studies because it often confounds the endpoints beyond the point of switching. Including it needs meticulous planning to avoid misinterpreting the efficacy data later.
Design biases and the size of their distortion
A trial described as randomised, controlled and blinded can still be biased by subtle design details that are easy to miss, and bias can arise at any stage, from setting the clinical question through design, conduct, data management and analysis to reporting. Common design biases affect a trial's outcome in different ways and by different amounts. Multiple primary endpoints or multiple arms create multiple comparisons, which exaggerate drug effects and raise the false-positive rate. An inappropriate crossover confounds the drug's effect on survival from the point of crossover. Inadequate allocation concealment or sequence generation imbalances group sizes and baseline characteristics, so the arms are compared unequally. Drug effect estimates were larger by 10% to 17% in studies with inadequate than with adequate allocation concealment, by 7% in studies with inadequate than with adequate sequence generation, and by 7% to 13% in unblinded than in blinded studies. A control with a distinct safety profile or dose-modification regimen makes the comparison unequal. In analysis, drug effect estimates were larger by 17% when a modified intention-to-treat population was used in place of intention to treat, and studies that excluded patients showed more beneficial effects than those that did not. The distorted results are likely to misguide the inferences drawn in reporting, and may undermine regulatory decision-making.
Estimands
Not considering estimands in the study design is a design bias too. An estimand framework is necessary to underpin any Phase 3 confirmatory trial design, oncology included. For Phase 3 confirmatory oncology trials, the intercurrent events should be defined at the outset, and appropriate strategies for them determined according to the clinical questions of interest. Different strategies for the same event answer different questions. A well-designed framework reduces the risks of missing data, helps address the right question, ensures appropriate analyses and supports the interpretation of the results. Since other oncology design guidances do not refer to estimands, the oncology examples in the E9(R1) addendum are helpful:
- treatment switching as an intercurrent event, for which the clinical question of interest must be clear and appropriate strategies applied, with a published framework for overall survival under switching
- clinical events that may be intercurrent events whose occurrence or non-occurrence defines different populations of interest, which can occur with time-to-event endpoints
- an intercurrent event that is itself informative about the outcome, such as treatment discontinuation considered part of progression-free survival and incorporated into its definition as a composite
Reporting bias
Reporting bias is passive when it comes from biases in the design choices, and active when it is the kind writers consciously try to avoid, not always successfully. The most common reporting biases include publication bias, not publishing trials with negative outcomes; outcome reporting bias, reporting only favourable data or a subset of it, or even changing the primary endpoint in the report; and spin, reporting that emphasises the drug's benefit when the hypothesis test does not support it. In an analysis of randomised breast cancer trials, one fifth of the studies reported in ClinicalTrials.gov had their primary endpoint altered in the final report. A third were reported as positive by spinning the results towards other endpoints, and half were reported as positive on a primary endpoint whose test was not statistically significant. These figures may be only the tip of the iceberg. The implications of reporting bias, passive or active, could be profound for oncology drug development and public health. Incomplete and skewed reporting could mislead policymakers deciding on approvals, thereby misinforming healthcare providers and potentially jeopardising cancer patients' access to effective treatment.
What the writer can do
A writer facing these biases can do four things:
- Be proactive from start to finish. Guidelines for generic and oncology trials exist from planning and design to accurate and transparent reporting. A writer involved in planning the research strategy is encouraged to engage with the regulators to discuss the best design given the disease, the available comparators, the drug's known benefit-risk and the long-term plan for collecting clinical-benefit data.
- Equip yourself with the right knowledge. Ask whether the comparator is appropriate, whether the endpoints fit the design and answer the clinical question, what magnitude of clinical benefit is expected, and whether a Phase 3 confirmatory design includes some kind of estimand framework. If it does not, open that discussion with the medical expert and the biostatistician. Being able to identify biases throughout the trial helps the writer report it critically and clearly.
- Know the types of reporting bias, the circumstances in which they may arise and their implications, which makes for more conscious writing.
- Remember that responsibility for appropriate design is not the writer's alone. The writer may need to raise awareness where it might otherwise be lacking, but the design considerations and responsibilities lie with the sponsor, the medical expert and the biostatistician.
Writers need to be aware of the possible biases in every aspect of an oncology trial and skilled to tackle them, which is essential for clear and transparent writing. Appropriate design can and should be advocated for by a well-informed writer, but must be a cross-functional effort involving at a minimum the sponsor, the medical expert and the biostatistician.