Quality and reproducibility in preclinical research
Preclinical research quality affects confidence in findings used to select targets and advance drug candidates. Reproducibility depends on study design, research materials, experimental conduct, analysis and the availability of results for independent assessment.
Replication findings
In a researcher survey cited in a 2017 review, 52% described a significant reproducibility crisis, more than 70% reported failing to reproduce another scientist’s results and more than 50% reported failing to reproduce their own.
Almost two-thirds of Bayer validation projects conducted between 2007 and 2010 showed inconsistent results, including findings from prestigious journals. Amgen researchers confirmed findings from six of 53 landmark studies, or 11%. A 2017 review also reported venture-capital firms’ view that experiments in at least half of published studies did not give the same results in independent laboratories.
Robustness is particularly important when preclinical findings determine whether a candidate enters clinical testing. A meta-analysis of animal stroke models found larger effect sizes in studies with low quality standards, which the review’s authors interpreted as making candidates appear more promising than they were.
Sources of irreproducibility
Survey respondents identified publication pressure and selective reporting as major causes of irreproducibility. Funding, promotion and tenure depend on publication, while pressure for a simple, complete story can encourage selective experiments, hypotheses fitted to existing data and continued data collection until a desired significance level is reached. Competition and publication pressure may also lead to negligent control or reporting of experimental conditions.
Difficulties publishing negative results or findings contradicting established views can strengthen a hypothesis even when unpublished evidence challenges it. In reproducible preclinical cancer studies, investigators paid close attention to controls, reagents and complete datasets; unreproduced studies often lacked blinded analysis and presented only one experiment.
Errors contributing to irreproducibility have been grouped into four sources:
- Study design.
- Biological reagents and reference materials.
- Laboratory protocols.
- Data analysis and reporting.
A re-examination of 100 compounds previously identified as therapy candidates in an amyotrophic lateral sclerosis mouse model found that most failed to slow disease, including eight that had progressed to unsuccessful clinical trials. The 2017 review’s authors considered low standards in the original work a likely explanation: most reports lacked statistical models to reduce experimental noise and did not implement blinding or randomisation.
Quality practices
A 2017 review identified a lack of commonly accepted and followed standards for non-regulated preclinical research outside studies directly supporting marketing authorisation. Its authors proposed specialised Good Research Practice guidance focused on study design, unbiased conduct, statistical analysis and transparent reporting. They considered clinical-research quality practices a possible model for improving preclinical work.
For non-regulated preclinical work, the proposed improvements included the following:
| Area | Practice or gap identified in 2017 |
|---|---|
| Audit | Monitoring compliance with quality requirements was increasingly discussed and already used by contract research organisations offering preclinical services; routine audits could also interest funders. |
| Independent review | Cell-line and in vitro work lacked a mechanism for feedback on quality and relevance. |
| Records | There was no clear consensus on the full set of experimental parameters to record, laboratory notebooks were not standardised and data witnessing was often not implemented. |
| Qualified contributors | Incorrect data analysis supported a greater role for biostatisticians in planning and analysis, and medical writers could contribute expertise in guidelines and reporting practices. |
A 2017 discussion noted that most animal research undergoes peer or ethical review, offering a chance to detect bias before experiments begin. It cited Swiss licensing based on detailed protocols and harm–benefit analysis, while warning that applications might be approved on implicit confidence rather than explicit evidence of scientific rigour.
Good Cell Culture Practice provided principles for standardising and harmonising cell and tissue culture laboratory practice, while cell-line misidentification, contamination and genotypic and phenotypic instability remained concerns.
By 2017, international consortia had begun multicentre, randomised, blinded animal studies described as preclinical phase III trials. These allow larger samples and reduced bias, improving robustness and translational predictability. Combining centres with slightly different food, housing or other environmental conditions also helps examine whether treatment effects depend on those conditions and how reproducible the findings are.
Transparent reporting and publication
By 2017, some preclinical journals required full datasets, biological properties of samples and complete methodology under open-data policies. Transparency and Openness Promotion guidelines advised journals and funders on incentives for transparent planning and reporting. Reporting guidelines prompt disclosure of technical and statistical information and consideration of reproducibility, but need to accommodate diverse experimental designs and analytical techniques.
A 2017 commentary advocated routine, reliable reporting of research regardless of whether outcomes were positive or negative. Researchers may hesitate to publish preliminary findings that further testing will not reproduce, but early expert review can help identify promising characteristics and reasons not to pursue an agent. Tentative early reports may remain valuable even when later found to be incorrect. Publishing negative findings can refine research, prevent repetition of futile studies and support collaboration.
Findings that are not published may disappear from collective knowledge, leaving the original team alone aware of why an agent was pursued or dismissed. Thorough, honest and accessible circulation of preclinical data can explain repurposing and rejection decisions, and outside researchers may suggest a different route or prevent premature dismissal.
Preclinical findings may not recur in clinical trials, and even reproduced findings may come from unrepresentative samples or be limited by confounding. Reports should therefore be accurate, relevant, realistic and open to critical appraisal, attracting expert input without premature claims that mislead. Communication professionals can help present positive and negative data systematically and transparently, making implications understandable even though subsequent media interpretation is less controllable.
Publication bias in animal research
A stroke-animal-study analysis using Egger regression and trim-and-fill found substantial publication bias; estimated efficacy fell from 31.3% to 23.8% after adjustment. Other studies cited in the same 2013 account reported effect-size inflation of 30% attributable to publication bias. Selective publication prevents assessment of the full evidence and may overestimate treatment effects, particularly when unpublished findings differ from published results.
Reporting also enables pooled retrospective analyses that may improve understanding of translation. A thorough, systematic review of animal evidence has been recommended before proceeding to clinical studies.
Registration proposals
A 2012 proposal called on funders, journals, foundations and academic institutions to promote registration and reporting of preclinical research supporting clinical trials. Its scope was controlled in vivo animal studies testing toxicity, toxicology and disease response, whose structure resembles controlled clinical trials.
The proposal’s authors argued that disclosure could reduce inefficient translation, protect human volunteers from unnecessary trials and make better use of their altruism. Free access could benefit patients, institutions and research communities, while delayed or absent disclosure impedes learning from development failures. They also argued that researchers disclosing unfavourable findings otherwise face reputational and funding disadvantages relative to those withholding them, encouraging biased reporting that may mislead treatment and resource-allocation decisions.
Cost and compliance were recognised obstacles: the proposal cited a 2007 ClinicalTrials.gov budget of $3 million plus investigator expenses and warned that existing reporting and compliance problems could recur in preclinical registries. Its authors expected benefits to outweigh costs and recommended cheaper models such as genomic and microarray data deposition. Journals could require confirmation that a complete preclinical evidence summary existed in a public database, and registries could begin with modest steps to refine models and data requirements. By 2013, a registry for congenital muscular dystrophy experiments was available. The proposal anticipated better exchange between scientists and more efficient translation into clinical use.
In 2016, the FDA commissioner outlined an idea for a broadly accessible preclinical research database. A 2017 account described a generally negative initial scientific reaction, based on concern that such requirements could constrain innovation and spontaneous discoveries.
A literature review and 21 international key-informant interviews identified 130 strengths, weaknesses, facilitators and barriers relating to animal-study registries. All stakeholder groups considered that registries could improve study quality and refinement, reduce animal numbers and support meta-research; the findings could inform debate and registry pilots.
Registered reports are a distinct publishing approach: by 2017, an increasing number of preclinical journals offered peer review before data collection, based on the research question’s importance and methodological quality. Acceptance was assured unless quality-assurance or unresolvable reporting problems arose.
Exploratory and confirmatory work
Clinical and preclinical research are not directly interchangeable, because basic and preclinical work needs room for creativity and innovation while also building on reliable findings.
The 2017 quality review’s authors proposed accepting statistical testing and low quality standards in exploratory work if experimental procedures were transparent. For confirmatory work intended to establish robust, reproducible treatment effects, they called for proper design and the highest standards despite the time and resources required. They recommended treating studies supporting decisions to advance to animal or first-in-human testing as carefully as clinical trials.