Randomized trial reports
The journal report of a randomized trial is drafted against CONSORT, whose checklist fixes what each part of the report carries, from the title and abstract through the allocation, the flow of participants and the outcomes to the registration, protocol and funding; cluster and pragmatic trials add items of their own, and a journal may ask for more than the checklist does.
The manuscript structure and the CONSORT checklist
One manual's outline for a trial report covers the introduction, methods, results and discussion, and includes disclosures: the registration with its ClinicalTrials.gov identifier, any previous presentation, the funding statement and the authors' financial disclosures, and the authors' contributions. CONSORT is the guideline to use when reporting the results of a randomized controlled trial. The quality-control checklists that manual gives for randomized trials are CONSORT and its extensions for cluster randomized trials and for abstracts. The CONSORT statement was devised to improve authors' reporting by providing a checklist and a flow diagram. Its recommendations were revised in 2001 and updated in 2010.
CONSORT was the first reporting guideline of this kind to be developed. The CONSORT website has a navigation panel at the top and on the right of the page that reaches the checklist item for each part of the article, and each part follows much the same structure as the title, with an explanation and an example.
CONSORT 2010, then the most recent version, is widely published in journal articles, most of them free, and is also available from the CONSORT website.
Title and abstract
CONSORT 2010 asks for identification as a randomized trial in the title. The website's explanation of the title item notes that whether a report of a randomized trial can be identified in an electronic database depends to a large extent on how it was indexed, and its example title states that the trial was a cluster randomized trial.
The abstract item's explanation carries a table of everything the abstract must include, and it can be kept open or printed beside the draft and answered item by item, yes or no: authors, trial design, participants, interventions, the study objective, outcomes, how randomization was performed and, if blinding was performed, how, how many patients were randomized to each treatment and how many were recruited, how many were analysed for each variable, the results for efficacy and harms, a conclusion stating the general interpretation of the results, the trial registration number and the sponsor. The CONSORT checklist for abstracts narrows the outcome to the primary one, with a result for each group and the estimated effect size and its precision, and adds the trial's status. It describes the trial design with examples such as parallel, cluster and noninferiority, asks for the name of the trial register beside the registration number and for the source of funding, and for a conference abstract adds contact details for the corresponding author.
Introduction
The introduction has only two checklist items, against the many the methods carry: the scientific background and the study objectives. The explanation describes a movement from what is already in the literature, what the general problem is and the environment in which the study was performed, through why the study is important, to the research question that the objective then states. One manual puts the typical aim of a clinical study's introduction as summarizing, very concisely, a disease and its contemporary management, identifying gaps and other limitations in that management, and so motivating the need for the research and its aims, building to a clinical problem or deficiency in knowledge and to how the study is unique or of incremental value in addressing it. A clinical study report supplied by the trial's grantor, typically a life sciences company, is typically useful in outlining the subject matter the study addressed, but its introduction represents that company's point of view. Even when the introduction focuses on the clinical development of one medication, including phase 1 to 3 data on the grantor's investigational product, the writer should strive to keep it otherwise agnostic about individual treatments, as clinical practice guidelines often do with stepped-care algorithms built from classes of medication without emphasizing individual agents. Citing a landmark study that compares several therapies for the condition may offer a widely applicable frame of reference for the broadest range of readers and practitioners. In schizophrenia, for example, the CATIE study, which compared clinical outcomes with a range of first- and second-generation oral antipsychotics in chronic schizophrenia, had been cited more than 5,000 times since its publication in 2005 when that manual was written.
Design, setting and participants
The methods carry many items: the trial design and any change made to it during the trial, the participants, the settings, the interventions, the outcomes and any change from the protocol, how the sample size was calculated, whether an interim analysis was performed and how, any guidelines for stopping the study before its expected end, how the randomization sequence was generated and what type of randomization was used, how allocation was concealed and how randomization was implemented, blinding, similar interventions where they were allowed, the statistical methods, and any additional analysis such as one on a subpopulation, on different variables, or of a type not in the protocol's statistical methods.
The CONSORT 2010 checklist asks for the trial design, such as parallel or factorial, with the allocation ratio, for any important change to the methods after the trial began, with the reasons, and for the interventions in each group in enough detail to allow replication, including how and when they were actually administered. The methods should also say how treatments were given, including any rules for titration, and report any protocol amendments or violations, data safety monitoring board and interim analysis.
If the study objectives could not be specified in the introduction, the methods open with them. The setting is characterized briefly, with the number and locations of the study sites and the study dates, such as when the first subject was enrolled or randomized and when the last was followed up, while numbers of subjects at each site, where available, belong in the results. Detailing the setting matters especially for an international trial, because responses to assessments may vary with culture and certain demographic traits. The eligibility criteria should be checked for consistency with any product labeling in the countries where the sites are. The setting may include statements about ethics and the protection of patients' rights and safety, which are identified from the clinical study report or protocol.
Graphing a schema and timeline may help to summarize the study design, especially when it is complex and not the typical parallel-group, randomized, double-blind, placebo-controlled trial. One manual's example schema, for a crossover trial, runs through screening, randomization, a first treatment period, a washout, a second treatment period and follow-up. In a crossover design each subject serves as his or her own control, so the observations in one subject are not independent and paired statistical tests are warranted.
An opening methods sentence should describe the design of the trial, and can carry the setting, the comparison, the dates, the ethical approvals and the trial registration with its number. The website's example names six tertiary neonatal intensive care units and a double-blind randomized trial in infants comparing the clinical consequences of transfusing red blood cells stored for seven days or less against standard practice, conducted between May 2006 and June 2011, followed by the ethical approvals and the registration. It is one of several examples the website holds, and the guideline's owners welcome good examples sent to them as additions to the collection.
Allocation and blinding
The checklist gives separate items to how the random allocation sequence was generated, with the type of randomization and any restriction such as blocking or stratification; to how the sequence was implemented, with any steps taken to conceal it until interventions were assigned; to who generated it, who enrolled participants and who assigned them to interventions; and, if blinding was done, to who was blinded after assignment and how. The older single-, double- and triple-blind labels can still be used, but what the guideline requires is a statement of who was blinded to treatment allocation. The 2001 checklist also asked, where blinding was done, how its success was evaluated. The types of randomization a report may name include simple, block, stratified and covariate-adaptive randomization.
Outcomes and statistical analysis
The methods identify which endpoints were prespecified as primary, secondary, tertiary or exploratory and which were post hoc, with the schedule of the assessments. Each instrument is detailed so that the reader can understand the results: its range of scores, whether an increase or a decrease signifies improvement, any normative values, the threshold for a minimum clinically important difference, and test properties such as sensitivity, specificity and predictive values. The assessments distinguish hard clinical from physiological or surrogate efficacy measures, patient-reported outcomes, safety and tolerability. A summary table can give the assessment schedule by visit, and if words are short it can go in a supplementary online appendix.
Most of the methods section should go to the assessment measures, their schedule and their attributes. The outcome measures are organized by efficacy, primary first and then secondary, tertiary and exploratory, and then by safety and tolerability. Safety assessments are typically made at baseline and at or soon after the end of treatment, and include a medical history with concurrent conditions and medications, a physical examination, vital signs, laboratory panels and a 12-lead ECG, with blood pressure and pulse rate typically measured at each visit; adverse events can be elicited by open-ended questioning at each visit after randomization. The methods should say whether the efficacy measures are hard clinical endpoints, such as all-cause or disease-specific mortality, or surrogate variables; hard endpoints may warrant Kaplan-Meier survival or other analyses such as Cox models.
The methods define the intention-to-treat, per-protocol and safety populations. One manual defines the intention-to-treat population as every patient randomly allocated to a study treatment whether or not the treatment was received, the per-protocol population as those who completed the study according to protocol, "typically with perfect adherence", and the safety population as all subjects who were randomized and received one dose or more of study treatment. Its annotated CONSORT flow diagram adds that the intention-to-treat population ignores anything after randomization, such as protocol deviation, nonadherence or consent withdrawal, that the per-protocol population completed the protocol without a protocol violation, nonadherence or consent withdrawal, and that the safety population, as a subset of the intention-to-treat population whose treatment may include placebo or usual care, takes into account what happens after randomization. By the same account, the intention-to-treat population may help to avoid artifacts of crossover and of attrition through dropouts that are not missing at random, and the per-protocol population is often used to assess efficacy.
The statistical issues one manual's outline lists for the methods are attrition, with whether data are missing at random or completely at random, and the imputation method; type 1 and type 2 error and multiple comparisons, for example with Bonferroni correction; and whether analyses are parametric or nonparametric, which is decided by plotting the data, testing for skew and using the Shapiro-Wilk test to determine whether the distribution is normal. Care is needed in using a categorical responder or target-achievement analysis as the major null hypothesis: unless the responder criterion is widely recognized as biologically relevant, defining the endpoint as the percentage of patients above or below a threshold is not necessarily statistically valid, because it categorizes an intrinsically continuous variable, and if such an analysis is formulated, the report includes precision limits, such as a 95% confidence interval, around the threshold for response. A subgroup analysis should be disclosed as prespecified in the protocol, as adjusted for multiple comparisons, for example by Bonferroni correction, or as post hoc; subgroup analyses are intrinsically hypothesis generating. For a randomized trial it is often appropriate to state one or more null hypotheses, each with an alternative hypothesis. Questions that may help to inform discussions with the biostatistician include how the sample size was calculated, which typically needs the treatment effect size and variance from similar earlier trials, the α level, the power and whether the data are paired or independent; how the tests were chosen, which also depends on how normal or skewed the data are; which method imputed missing data, where last observation carried forward was relied on in the past and multiple imputation and other approaches are increasingly advocated; and which covariates were controlled, since the typical analysis of covariance, with treatment and study site as covariates, often leaves out other potentially influential baseline variables such as age, number of comorbidities, and duration or severity of disease. The statistical methods of the clinical study report, the protocol or the statistical analysis plan are reviewed to understand why particular approaches and tests were chosen, and the kick-off meeting may be a chance to ask the biostatistician about them.
Participant flow and baseline characteristics
The results start with how many patients were screened for inclusion and how many randomized, who took the drug, who completed the expected length of the trial, who dropped out and for what reasons, when recruitment started, and, if the trial stopped before its expected end, why it was interrupted. The flow of participants runs from the number assessed for eligibility, with exclusions and their reasons, through the number randomized and allocated to each group and receiving the allocated intervention, and the losses to follow-up and discontinuations with their reasons, to the number analyzed and excluded from analysis. It can be presented either as the flow diagram on the CONSORT website or as a table, both offered from the website's menu. Ideally, a patient-disposition flowchart gives the number of potential participants screened as well as the number randomized, which conveys how selected the population was and any potential difficulty in recruiting and enrolling subjects. One sequence for the results of a randomized trial runs from patient disposition through baseline characteristics and efficacy to tolerability and safety.
The baseline characteristics of the patients come next, as a table of the baseline demographic and clinical characteristics of each group.
Outcome results and harms
After the flow and the baseline data, the checklist asks for the results and any ancillary analyses, binary outcomes where a response was present or absent, and last, and in most cases most important, the harms. Both positive and negative findings are reported. CONSORT 2010 asks, for each primary and secondary outcome, for the results for each group and the estimated effect size and its precision, such as a 95% confidence interval. The comparison giving rise to each p value should be clear. Results in the narrative text should agree with those in the tables, figures and abstract, and with the raw or analysed data in the clinical study report or other source documents. Data from post hoc analyses are designated as such. Normally distributed data are reported with the mean, standard deviation, 95% confidence interval, p value and the test that generated it, and skewed data with the median and interquartile range or another measure of variability. When an analysis of covariance controls for baseline covariates, least-square means of the changes from baseline are typically compared. For binary outcomes, CONSORT 2010 recommends presenting both absolute and relative effect sizes, and the 2001 checklist asked for results in absolute numbers when feasible, such as 10/20 rather than 50%. The original data tables should be requested from the statistician in an editable format, because rekeying the data can introduce errors, and once the tables are set they are reviewed with an editor for duplicated rows or columns, which often indicate transcription errors.
Discussion
The discussion has three items: limitations, generalizability and interpretation. CONSORT 2010 asks for the trial's limitations, addressing sources of potential bias, imprecision and, if relevant, multiplicity of analyses, and for an interpretation consistent with the results that balances benefits and harms and considers other relevant evidence.
One manual's outline lists as typical limitations of a randomized trial its limited ecological validity, or generalizability to real-world treatment settings, a highly selected patient population in which tolerability, adherence and overall treatment response may be overestimated, and protocol-based, inflexible treatment regimens. Randomized trials are typically not powered to determine differences between treatments in infrequent adverse events, nor long enough to determine differences in safety parameters. The same outline lists as strengths that random allocation, blinding and placebo control largely preclude bias or confounding, the high quality of patient-level data, and guidance on endpoints from the FDA. One question for the discussion is how well balanced the groups were at baseline and how effective randomization was. Another is how representative the population was of readers' practices and of larger populations, judged by comparing the baseline characteristics with epidemiologic statistics or with those of related studies.
The discussion recaps the key findings and probes their implications, asking whether the study objectives were met and, if not, why not, whether the findings are clinically as well as statistically significant, and, where prespecified subgroup analyses were conducted, whether any patient segment derived special benefit or experienced more adverse consequences. It compares the data with similarly designed studies, with other recent ones from the past one to five years, and with pivotal studies of the same agent or of related agents in its pharmacologic class. It also asks whether there are alternative explanations for the findings and what future studies are warranted to evaluate, corroborate, extend or refute them.
Registration, protocol and funding
CONSORT 2010 asks for the registration number and the name of the trial registry, where the full trial protocol can be accessed, if available, and the sources of funding and other support, such as the supply of drugs, with the role of the funders. If the study was registered, its identifier, such as a ClinicalTrials.gov NCT number, is given.
Cluster randomized trials
A cluster trial report gives the rationale for using a cluster design, the eligibility criteria for participants and for clusters, and whether the interventions, objectives and outcomes pertain to the individual level, the cluster level or both. It describes how the total sample size was determined, including the method of calculation, the number of clusters, the cluster size and a coefficient of intracluster correlation with an indication of its uncertainty, and specifies that allocation was based on clusters rather than individuals. Its statistical methods indicate how clustering was taken into account.
The flow of clusters and of individual participants is reported through each stage, from random assignment through treatment and completion to analysis, and baseline information is given for the individual and cluster levels as applicable. Each primary and secondary outcome is reported for the individual or cluster level as applicable, with the estimated effect size and its precision and, for each primary outcome, a coefficient of intracluster correlation, and generalizability is discussed for individuals, clusters or both, as relevant.
Pragmatic trials
The PRECIS tool of 2009 describes a pragmatic and an explanatory approach to each domain of a trial, including participant eligibility, the flexibility of the experimental and comparison interventions and the expertise of the practitioners who apply them, follow-up intensity, the primary outcome, participant compliance, practitioner adherence and the analysis of the primary outcome. At the pragmatic end, everyone with the condition of interest is enrolled regardless of anticipated risk, responsiveness, comorbidities or past compliance, there are no formal follow-up visits and administrative databases are searched for outcomes, and the analysis includes all patients regardless of compliance or eligibility. At the explanatory end, selection is restricted step by step to high-risk, highly responsive, compliant individuals, follow-up is far more frequent than in routine practice, and an intention-to-treat analysis may be supplemented by a per-protocol analysis or one restricted to compliers.
The background of a pragmatic report describes the health or health service problem the intervention is meant to address and other interventions that may commonly be aimed at it. Its eligibility criteria are framed to show how far they include typical participants and, where applicable, typical providers, institutions, communities and settings of care. It describes any resources added to or removed from usual settings to implement the intervention, whether the intervention and its delivery were standardized or allowed to vary, the comparator in similar detail, and why the chosen outcomes and, when relevant, the length of follow-up are important to those who will use the results. If the sample size was calculated from the minimally important difference for the target decision makers, the report says where that difference came from, and if blinding was not done or not possible, it explains why.
To the standard results items, the pragmatic extension adds the number of participants or units approached, the number eligible and the reasons for non-participation. For generalizability, it asks for the key aspects of the setting that determined the results and for possible differences in other settings where clinical traditions, health service organization, staffing or resources may vary from those of the trial.
Journal requirements beyond the checklist
By 2019, journals and their reviewers were increasing their requirements for publishing randomized controlled trials. The editors' aim was to be sure that authors were telling the truth about their observations. Some requirements had become routine: recording the protocol; supplementing the article with all key trial data; including the completed CONSORT flow chart and adhering to the CONSORT guideline; structured summaries with key points or highlights; and sometimes an abstract for the public and a closing sentence headed "Conclusion and Relevance" at the end of the abstract. Others depended on the journal: reporting effect sizes with 95% confidence intervals instead of P values, the 0.05 significance level being a historical tradition rather than a rationally established cut point; reporting how many secondary outcomes the protocol specified and exactly how many are reported in the manuscript; being explicit about post hoc analyses; and providing a visual abstract. The objective of these requirements was transparency and the fight against the crisis of reproducibility.
Worked examples
One manual works through a hypothetical report of a randomized trial of an investigational anticholinergic agent for a hiccup syndrome as an exercise; it cites references and calls out graphics that an actual manuscript would likely include but leaves them out to keep the exercise brief. Its introduction moves from disease definitions and epidemiology, normal physiology, and the disease state to the gaps in research and the study rationale, and then to the thesis statement and objectives. Its methods cover the study design and setting, participants, ethics, interventions, assessments and outcome measures, safety and tolerability, and statistical methods. Its results run from patient disposition through baseline characteristics and efficacy to tolerability and safety.
Its discussion offers a potential reason for a somewhat unexpected nonsignificant finding on severity: the assessors' ratings of noise and smell varied widely, and the reliability of the ratings, by Cronbach's α and the ω coefficient, was low. Its limitations note that the study was underpowered, and at 12 weeks arguably too short, to discern differences in adverse events and safety signals occurring in fewer than 1 in 100 to 1 in 1,000 subjects, with long-term safety left to an extension study. Its conclusions call for future studies in larger and more heterogeneous populations to reject, corroborate or extend the findings.