Nonrandomized intervention reports

A report of a nonrandomized evaluation of an intervention is recommended to show how units were assigned to interventions and what participants actually received, what was done to minimize potential bias from nonrandom assignment and to control for baseline differences, and how alternative explanations bear on the results.

The TREND checklist

The TREND statement of 2004 has a 22-item checklist, some of whose topics and descriptors are added to, modified from or further emphasized relative to the CONSORT statement; those not in CONSORT are relevant to behavioral interventions evaluated with nonrandomized experimental designs. ¹ The checklist is in the public domain, and in 2017 it was available on a website of the US Centers for Disease Control and Prevention. ¹

The checklist recommends that a report:

1. Give information in the title and abstract on how units were allocated to interventions; a structured abstract is recommended.
2. Give the scientific background, the rationale for the study and the theories used in designing the intervention.
3. State the eligibility criteria for participants, including those at different levels of recruitment or sampling; the method of recruitment, with the sampling method if a systematic sampling plan was used; and the recruitment setting.
4. For each condition, describe the intervention intended and how and when it was actually administered, including its content, delivery method, grouping of participants, deliverer, setting, number of sessions, episodes or events and how long they were intended to last, the time intended to deliver the intervention to each unit, and activities to increase adherence.
5. State the specific objectives and hypotheses.
6. Define the primary and secondary outcome measures clearly, and describe the methods used to collect data, any methods used to enhance the quality of measurements, and the psychometric and biometric properties of validated instruments.
7. Say how the sample size was determined and, when applicable, explain any interim analyses and stopping rules.
8. Name the unit of assignment and the method used to assign units to conditions, including any restriction such as blocking, stratification or minimization, and the measures taken to help minimize potential bias from nonrandom assignment, such as matching.
9. Say whether participants, those administering the intervention and outcome assessors were blinded to assignment and, if so, how blinding was accomplished and how it was assessed.
10. Describe the smallest unit analyzed to assess intervention effects, and where the unit of analysis differs from the unit of assignment, explain the method that accounts for it, such as adjusting standard error estimates by the design effect or using multilevel analysis.
11. Give the statistical methods used to compare groups on the primary outcomes, including complex methods for correlated data, the methods used for additional analyses such as subgroup and adjusted analyses, any method used to impute missing data, and the statistical software used.
12. Show the flow of participants through each stage of the study by condition, for which a diagram is strongly recommended: the numbers screened, found eligible or not eligible, declining enrollment and enrolled; the numbers assigned to each condition and receiving each intervention; the numbers completing and lost to follow-up; and the numbers included in or excluded from the main analysis.
13. Give the dates defining the periods of recruitment and follow-up.
14. Describe the baseline demographic and clinical characteristics of each condition, including those relevant to specific disease prevention research, and compare participants lost to follow-up with those retained, overall and by condition.
15. Give the data on baseline equivalence and the statistical methods used to control for baseline differences.
16. Give the number of participants included in each analysis for each condition, particularly where the denominator changes between outcomes, and state the results in absolute numbers when feasible; say whether the analysis was intention to treat and, if not, how noncompliers were treated in it.
17. Report each primary and secondary outcome by condition with its effect size and confidence interval, including null and negative findings and the results of any prespecified causal-pathway tests.
18. Summarize the other analyses performed, including subgroup or restricted analyses, labelling each as prespecified or exploratory.
19. Summarize all important adverse events or unintended effects in each condition, with summary measures, effect size estimates and confidence intervals.
20. Interpret the results in light of the study hypotheses, sources of potential bias, imprecision of measures, multiplicative analyses, and other limitations or weaknesses of the study; discuss them in terms of the mechanism by which the intervention was intended to work or alternative mechanisms or explanations, the success of and barriers to implementing the intervention and the fidelity of implementation, and their research, programmatic or policy implications.
21. Discuss the generalizability (external validity) of the findings, given the study population, the characteristics of the intervention, length of follow-up, incentives, compliance rates, the specific sites or settings, and other contextual issues.
22. Give a general interpretation of the results in the context of current evidence and theory.

^{2, 3, 4, 5, 6, 7, 1}