Balanced interpretation of findings
Balanced interpretation weighs supportive and countervailing evidence and limits conclusions to what the study's methods, observations and context can support.
Evidence on both sides
Fair balance requires congruence between the clinical problem and research objectives, between objectives and methods, and between findings and conclusions. A discussion must explicitly weigh evidence both for and against its argument, avoiding conjecture, overinterpretation and an excessively defensive or argumentative tone.
A sponsor's clinical study report is an important starting resource but represents the sponsor's perspective on the disease and its management. Reading widely, including clinical guidelines and high-quality reviews, and speaking with the corresponding author and other scientists can balance that perspective. A mixed literature should be described as mixed: “some, but not all, studies” preserves contradictory findings that a simple “studies show” can conceal. Cochrane systematic reviews and meta-analyses can help establish the wider evidence landscape without cherry-picking.
An account of a single product can consider evidence on treatment classes and technologies and their place in clinical practice guidelines, rather than treating the product in isolation. Coauthors who have served on guideline panels can explain disputed issues and the perspectives of potential reviewers.
Reconciling divergent findings
Differences between randomised and observational findings can arise from several sources identified by an ISPOR best-practices panel.
- Different aims and endpoints, such as efficacy in randomised trials and effectiveness in observational studies.
- Different study populations.
- Confounding in observational studies, particularly medication adherence, patient copayments and confounding by indication.
- Poor-quality patient-level data in observational studies.
These differences may need reconciliation when comparing findings with published values or pooling studies, even when they do not arise in reporting one study alone. Interpretation should make tensions explicit: particular versus general findings, controlled versus observational designs, association versus causation, prospective versus retrospective evidence, stakeholder perspective and frequentist versus Bayesian reasoning.
One writer's US university-hospital example shows why a setting can carry competing influences: patients may receive advanced care while urban recruitment brings people with fewer resources or lower health literacy to adhere to treatment.
Proportionate conclusions
Conclusions should draw statistically and logically sound inferences rather than simplify away scientific complexity or conform to a predetermined message. Wording should reflect the study's design, population, methods and observations: a finding that participants receiving A had more favourable responses than those receiving B is narrower than a general assertion that A is more efficacious.
Ambiguous claims need their outcome specified: “less bleeding” leaves unanswered whether frequency, severity or both decreased. Reduced dosing frequency or a different administration route does not necessarily establish convenience or adherence, which also depend on patient characteristics, tolerability and costs. A longer half-life may prolong adverse effects as well as benefits, so it does not establish improved patient-reported outcomes.
Qualitative descriptions such as “robust” or “good” efficacy need an anchor in normative or clinical data; patient-reported changes can be termed clinically important when they meet published minimal-clinically-important-difference thresholds. Unanchored superlatives should be avoided.
Interpretation should return to the prespecified question: whether objectives were met, which results were expected or unexpected, what alternative explanations remain and what further research could challenge or extend the findings. Considering how an impartial reviewer, an opposing scientific school or a competing treatment's developer might view the findings can expose assumptions overlooked by an invested research team. Apparent patterns may also reflect chance or biological variation, which enthusiasm for one's own data can obscure.
Association and causal inference
Observational associations should generally be reported as associations rather than effects, since correlation does not itself establish causation or its direction. A finding that treatment A was associated with a lower five-year event rate than treatment B does not establish that A reduced that rate.
Two logical errors confuse necessary and sufficient conditions. If rain makes a car wet, observing a wet car does not establish rain, because a car wash could explain it; this is affirming the consequent. Nor does absence of rain establish a dry car, since the car wash remains possible; this is denying the antecedent. Reliance on an expert's assertion in place of evidence is the related danger of ipse dixit.
An exposure preceding an outcome is not by itself proof that the exposure caused it. Likewise, an association between a low surrogate measurement and greater disease risk does not establish that raising the measurement through treatment will reduce risk.
Albert Michotte's experiments in the 1940s illustrated how observation can become causal belief: viewers seeing one ball touch another and the second move inferred that the first caused its motion. A 2011 critique argued that searching for small correlations without examining secondary and tertiary interactions could miss how a biological system works as a whole. Readers of that critique countered that sufficiently powered, double-blind studies and rigorous experimental design had exposed incorrect causal assumptions in its own examples.