Analysing real-world data
Real-world data cannot be analysed the way a randomised trial is, because the design of a trial supports assumptions that routine data do not meet. Regression, propensity scores, instrumental variables, causal-inference models, Bayesian methods and machine learning each answer that problem differently, and a correct description of any of them rests on what the method does and on what it can and cannot claim.
Why real-world data need different methods
An analysis of real-world data often needs different statistical methods from those of a typical clinical trial, especially a randomised one, because most standard methods assume that the groups compared were reasonably well balanced in known and unknown potential confounders before the intervention. They also assume that follow-up stays comparable across groups, except for differences directly attributable to the intervention itself. Randomisation can help with the first assumption, and principled adherence to a well-developed protocol that strives for controlled and uniform follow-up can help with the second, so a trial's tests of group differences can potentially be interpreted causally. An analysis of real-world data lacks both of these and other safeguards, so it needs statistical methods sensitive to such issues. Several approaches developed primarily for data outside randomised trials can be useful when analysing real-world data.
Selection bias, confounding and variable data collection and endpoint definitions concern every design, but they are even more prominent with real-world data, and anticipating and addressing them is crucial for the integrity, validity and applicability of the findings. Selection bias, confounding and inconsistent data collection, as inherent challenges of real-world data, require specialised analytical approaches, such as propensity score methods, Bayesian techniques and causal-inference models.
Common strategies for reducing bias
Common strategies for reducing bias in real-world studies include the following.
- Restriction applies strict inclusion and exclusion criteria to make the patient population more homogeneous.
- Stratification divides the study population into subgroups by potential confounding variables.
- Regression analysis adjusts statistically, with multivariable models that take confounding factors into consideration.
- Propensity score matching effectively mimics randomisation by creating treatment and control groups balanced on specific baseline variables.
Sensitivity analyses, which test the potential influence of unmeasured confounders, are also used to demonstrate that the results are robust.
Instrumental variable analysis, like propensity score matching, is an advanced analytical technique that can help address confounding and bias in real-world studies.
Regression
Regression analysis is a key statistical tool for exploring and quantifying relationships between variables, such as treatment exposure and clinical outcome. In epidemiological studies and real-world analyses, linear and logistic regression, repeated-measures analysis and Cox proportional hazards models are commonly applied to control for confounding when characterising relationships between exposure and response. By adjusting for patient demographics, comorbidities and other covariates, regression can help isolate the effect of a specific variable of interest, enhancing the validity of conclusions drawn from non-randomised, real-world settings. Its limitations with real-world data include confounding by unmeasured or misclassified variables, model misspecification, selection bias and missing data, but regression, especially when used properly, remains an essential and widely used tool in real-world analyses.
Propensity scores
Propensity score methods were developed to help create groups of patients balanced on observed baseline characteristics in observational studies. They generally fall into two categories: matching, which tries to pair patients with similar characteristics across treatment arms, and inverse probability of treatment weighting, which weights patients by their propensity scores to create a pseudo-randomised population. Either way, the scores build pools of patients that appear similar in the distribution of covariates, irrespective of the treatment they went on to receive. So, if properly executed, an analysis of outcome differences that incorporates propensity scores could help to evaluate treatment differences adequately in real-world data.
Causal-inference models
Causal inference is a rich and long-standing field in which numerous methods and theories have been developed. Its methods are rarely a standard topic in academic statistics programmes, even in advanced postgraduate study, and like many advanced methods they should be handled only by experienced professionals. Many of its approaches examine relationships between variables or concepts not directly observable in the data, or try to estimate causal relationships in the presence of confounding.
Structural equation modelling, one such method, examines relationships between observed variables and latent constructs, such as depression or quality of life, which cannot be observed directly and are inferred from other measurable variables. It combines factor analysis and regression, and can be drawn as a diagram of hypothesised causal paths among the variables. One study used it to examine how HIV-related symptoms were related, directly and indirectly, to adherence to antiretroviral therapy, and whether depression mediated the relationship. General HIV-related symptoms and gastrointestinal symptoms were each directly associated with more depressive symptoms, which were in turn directly linked to lower adherence. Neither kind of symptom had a direct effect on adherence, only an indirect one through depressive symptoms.
Bayesian methods
Bayesian methods, which make inference by updating prior beliefs with data, have found numerous applications in the design and analysis of clinical trials. They bring prior external information, such as evidence from completed trials or expert opinion, into the current analysis, which allows more robust inference even when data are limited or heterogeneous. With real-world data they can account for missing data by predicting unknown values from the available data, as in multiple imputation, adjust for confounding through Bayesian propensity score methods, and account for variability across populations with hierarchical models, as in pragmatic trials. They can also update inferences as new data arrive, which makes them particularly valuable for ongoing studies and real-time decisions.
Machine learning
Machine learning can be particularly useful with real-world data, because the datasets are often large and complex. Its results include improved insight through supervised learning, such as random forests, or help with predictive modelling and treatment-effect estimation. Unsupervised learning such as clustering, or dimensionality reduction such as principal component analysis or lasso regression, can help to identify patterns within patient populations.