Missing data
Every clinical study loses some of the observations it set out to collect, and a writer reporting one has to understand why the report must account for them, how to judge their quantity and their pattern, what the design can do to limit them, which assumption about the missingness an analysis makes, why the analysis cannot fall back on the participants who completed, and the bias each handling method risks.
Why reports account for missing data
Missing data are ubiquitous in clinical studies, and both their extent and their structure can introduce bias into study results and lead to wrong conclusions. Sooner or later, usually sooner, anyone who does statistical analysis runs into problems with missing data. They can look like a problem for the data manager and the statistician rather than for the writer, but every study written up in a publication or a study report will have dealt with them, and the way those conducting the study handled them can have far-reaching consequences for the validity of the conclusions. Because the complexity of a clinical study means everything is related to everything else, missing data are linked to study design, patient retention, data analysis and the conclusions that can be drawn, so every scientific report about a clinical study must take note of the extent to which data are missing and of how that has been handled. Medical writers should be aware of the extent of missing data and should describe the methods used to deal with it.
In an ideal trial every patient would come to every visit, take the medication each day at the right time and undergo every procedure as planned. No equipment would fail, every shipment would arrive, every laboratory analysis would be done under the specified conditions and no sample would be lost. No investigator or patient would move or decide to leave the study, and nobody would have an accident, fall ill or die during it. Only then would the writer be absolved of having to talk about missing data, and because the real world is never perfect, the issue will invariably arise.
How much is missing
It cannot be assumed that all the data will be obtained for all the patients in a clinical study, and whether that is a problem depends on the quantity and nature of the missing data. The more data are missing, the shakier the results and conclusions become. It is very difficult to say when a critical limit has been reached, because the size of the study, the indication, the magnitude of the difference between treatments, and the frequency and nature of the assessments all have to be considered. Where a trial tests for a difference in outcome events, that is, medically important events such as heart attacks in patients at high cardiovascular risk, even a small number of missing data may be important. Where outcome data are missing for a sizeable proportion of patients, the whole trial may become invalid.
The pattern of missingness
When the pattern of missing data differs between the treatment groups, it is likely to introduce bias in the interpretation of results. Some data go missing by pure chance, as when a patient misses a visit because her car broke down, or leaves the study because he is moving for his wife to take a job in another region. In a randomised trial comparing two treatments, such chance events should not be much of a problem provided they are rare, because they would be expected with similar frequency in both groups and so introduce no bias. Missingness related to the treatment or to the outcome leaves the analysis on very difficult ground, as when one group has a much higher dropout rate than the other, which may happen for many reasons, such as adverse events, lack of efficacy or unknown reasons. It is often difficult to know whether data are missing by chance or because of the treatment: when a patient taking a drug that may cause dizziness has a traffic accident on the way to the clinic and ends up in hospital, the accident may be either.
When a new drug is compared with placebo, adverse events may cause dropout in the active arm while placebo patients carry on, which risks underestimating the treatment effect. Conversely, good efficacy and no tolerability issues may keep active patients in while placebo patients leave because they see no improvement, which risks overestimating it. Randomisation aims to give the two groups similar characteristics at the start of the study, so if many patients in one group but not the other withdraw, the groups may no longer be comparable at the end, which is a serious conceptual problem. If a sizeable proportion of the active group drops out because of tolerability issues, that group has not only more missing data but also different people at study end: exposing patients to the drug unintentionally selects those able to tolerate it, so the study ends by comparing the placebo group, with all its initial demographic and disease characteristics, against an active group of tolerators whose baseline characteristics may be quite unrepresentative of the initial population. That makes it very difficult to draw any conclusions about the efficacy or safety of the treatments, because the statistical techniques commonly used to analyse clinical trial data require comparing randomised treatment groups.
When reporting clinical studies, medical writers need to be alert to signs that missing data are not due to chance and so have the potential to cause bias: differences between the treatment groups in the proportions of patients with missing values or in the reasons for withdrawal, and clusters of withdrawals or missed visits around particular points in time. The tables of patient disposition can be a starting point, and where they show a problem it is advisable to ask the statistician for further information on the missing data.
Limiting missing data by design
No amount of statistical expertise can make up for the absence of real data, so preventing missing data in the first place needs to be a top priority, and a number of measures at the protocol stage can help limit their quantity. Most importantly, a trial needs to interfere only minimally with the normal life of its participants. Study visits should be scheduled at convenient times and should not take too long, minimising the number of visits and assessments where possible is likely to help retain patients, and generous visit windows make it easier to fit visits around other commitments. The longer the follow-up, the more patients are likely to withdraw, so a short follow-up period, at least for the primary endpoint, can help minimise the impact of missing data. Endpoints that are difficult or time-consuming to measure, or that require invasive procedures, tend to produce a high quantity of missing data, so endpoints that are easy to measure are likely to reduce it.
Missing data that arise from adverse events or lack of efficacy are especially problematic, because they tend to be associated with a particular treatment and so risk biasing the results. Flexible dosing can minimise withdrawals over tolerability. Withdrawals for lack of efficacy are a common problem among patients receiving placebo, and an add-on design, in which patients receive active treatment or placebo in addition to standard treatment, can help avoid them. Where a patient does have to discontinue study treatment, the sponsor should ask permission to continue collecting data from them, and the study should be planned so that discontinuing treatment does not necessarily mean withdrawing from the study. During the trial, engaging participants with clear explanations of its purpose and procedures will most likely reduce the number of patients who withdraw. Missing data can never be prevented altogether, so to collect data from enough patients for valid conclusions, it is important to consider the likely number of missing values when planning the trial and to allow for them when calculating how many patients to recruit.
The choice of investigators and sites can take into account their track record of following up patients and collecting complete data. Sites can be chosen that are conveniently located and easy to reach, with facilities that make visits easier, such as plentiful parking and on-site child care. Investigator training and communication with patients can stress the importance of staying in the trial until the end.
Assumptions about why data are missing
Statisticians categorise the reasons for missing data in a theoretical framework that distinguishes data missing completely at random (MCAR) from data missing at random (MAR) and data missing not at random (MNAR). Assuming that data are missing completely at random means assuming no systematic relationship between the outcomes, observed or unobserved, on the variable being analysed and the probability of dropout. Missing at random is usually more plausible and applicable than missing completely at random, because “MAR is valid in every case when MCAR is valid, but MCAR is not always valid when MAR is valid”.
Analysing only the completers
Restricting the analysis to the patients who completed is not a good idea. In a type 2 diabetes trial whose primary endpoint is change from baseline in HbA1c, five patients show why: one completer, one who withdrew at the second visit, one with no baseline value, one who died after the third visit, and one who did not attend all visits. Only for the completer can the change from baseline be calculated easily. The data do not say why the others are missing, which is common in clinical trials: broad reasons such as adverse event or lack of efficacy may be recorded, and the reason a patient died, but patients lost to follow-up or missing some visits may have no detailed reason recorded. Patients may miss visits because of the severity of their disease, because of adverse events, or because of chance events having nothing to do with their health, while those who attended every visit could be the younger patients with fewer comorbidities, fit and mobile enough to make every visit. So an analysis of completers, or observed cases, may select patients who are not representative of the population as a whole, and disregarding all patients with incomplete data would not only risk bias but also lose a lot of valuable information. Whatever the reasons for their missing data, the patients with incomplete data may be representative of actual clinical practice.
Imputation
A number of statistical methods exist for handling missing data, and the risk of bias in a particular situation varies with the method chosen. Imputation includes substituting a value for a missing observation, and its common forms are last observation carried forward (LOCF), regression imputation and hot-deck imputation. For ethical and economic reasons it would be wise to use all the data gathered, and LOCF, used for many years, is a very simple way to do so: each missing item is filled in, or imputed, with the last observation obtained at an earlier time point. After LOCF, the change from baseline to study end can be calculated easily, from the first visit for the patient with no baseline value. Baseline observation carried forward (BOCF) is a similar method that carries the patient's baseline value instead.
Although appealing in its simplicity, LOCF is likely to introduce bias and may even lead to wrong conclusions. In depression, some patients improve spontaneously, so if many patients in the active arm drop out because of adverse events, LOCF keeps values recorded before all their spontaneous improvement had a chance to surface, and the treatment effect is likely underestimated. In a condition that worsens over time, placebo patients go on worsening to study end, while active-arm patients who leave early because of adverse events keep an earlier, better score than they would have had at study end, so the treatment effect is likely overestimated. Because of that potential for bias and incorrect conclusions, regulators and leading statisticians urge clinical researchers to stop using LOCF. LOCF is restrictive because it assumes that the data from lost patients are missing completely at random, and that nothing changes between the last available visit or observation and the end of the study.
Other methods include listwise deletion, which typically works more effectively, and with lower bias, when data are missing completely at random rather than only missing at random. Pairwise deletion, or available-case analysis, is appropriate for linear models such as factor analysis and linear regression, and like listwise deletion it can give biased estimates when data are missing at random rather than completely at random. Dummy variable adjustment is used in regression analysis when one or more independent variables are missing: it assigns a constant dummy variable and regresses the dependent variable on both the independent variable and the dummy, though it may give biased estimates of coefficients. The maximum-likelihood method, which includes least-squares linear regression, chooses as estimates the values that, if true, would maximise the probability of observing what has in fact been observed. It works effectively for certain models, linear and log-linear, but not for others, such as ordered logistic regression or Cox proportional hazards.
Multiple imputation (MI) does not fill each gap with a single replacement value. It uses the data collected from all patients, complete or not, to model the distribution of the missing data, and uses the model to generate a series of values for each missing observation. The result is a series of data sets, each holding the observed and imputed values, and the prespecified analysis can be run on each and the results combined into one overall estimate of the treatment effect. MI surmounts many of the limitations of maximum likelihood, and it is considered variable rather than deterministic: because the process includes random variation, it is less likely to underestimate covariances and the values of variables with missing information. MI includes single random imputation and multiple random imputation. Single random imputation makes a random draw from the residual distribution of each imputed variable and adds random numbers to the imputed values, after which conventional formulas can be used to calculate variances and covariances. Multiple random imputation repeats the process to produce several completed data sets whose estimates differ slightly, and that variability across imputations can be used to adjust the standard errors upward.
Mixed-effects models for repeated measures
Mixed-effects models for repeated measures (MMRM) are a family of analyses frequently used when the same continuous outcome is measured repeatedly at different time points, and they make it unnecessary to fill in missing values with substitute values. They combine the information available for each patient with missing data with the information from patients with complete data to predict what the missing responses would have been. For example, if a patient improved a little early in the trial and withdrew after three weeks, while most of the same group improved more in the first three weeks and went on improving to the end, the model assumes the withdrawn patient would also have continued to improve, but by less than average. Unlike single imputation such as LOCF and BOCF, MMRM and MI use all the available information for each patient rather than one value, and they have been shown to produce much less biased estimates than LOCF. The traditionally used LOCF method is problematic in many ways, and it is better to use a method that reduces bias, such as MI or MMRM.
Mixed-effects models can reduce the problems associated with missing data: they essentially treat each observation as a single data point, so if one observation is missing, the participant's data from other time points are not lost. MMRM analyses are data conserving, in that a subject's observations are not all lost on consent withdrawal or attrition. An MMRM analysis allows for the possibility that observed outcomes are related to the probability of dropout, which is consistent with missing at random and not with missing completely at random. Mixed-effects models are also often useful in real-world evidence studies with irregular visit schedules or observation points, because for comparing subjects or groups it does not matter if one individual has a second assessment at 3 weeks and the next at 3 months while another has them at 3 months and 6 months.
Sensitivity analyses
No single method produces unbiased results in all circumstances, so as well as choosing a method appropriate to the situation, it is important to test the robustness of the results with sensitivity analyses that use missing-data methods other than the primary one. Those methods should rely on different assumptions from the method used in the primary analysis. If MMRM is primary, the sensitivity analysis might include LOCF, BOCF and MI. It might also include MI and sophisticated modelling techniques that do not make the same assumptions as MMRM about the nature of the missing data. If LOCF is primary in a long-term trial with many patient withdrawals, a sensitivity analysis of completers only would help. Similar results give confidence that the results are not being unduly influenced by the method used for handling missing data, while substantially different results mean the issue needs further investigation and discussion.
Reporting how missing data were handled
A methods section should answer how missing data were handled, that is, by which imputation method.