Real-world evidence for medical devices
Real-world data show how a device performs in routine care, across patients and settings that a clinical investigation does not reach, and they feed a device's post-market clinical follow-up, its clinical evaluation and its submissions. The parts of the concern are why investigations alone are not enough, what real-world data add and the gaps they close, the sources they come from, cases in which one closed an evidence gap, their use under the EU regulations, the US framework, a collaborative initiative for in-vitro diagnostics, statistical methods for setting endpoints from them, and AI-driven meta-analysis of the growing pool of clinical data.
Why clinical investigations are not enough
The use of real-world data for medical devices has drawn increasing attention from regulatory agencies, industry stakeholders and the scientific community. Devices from in-vitro diagnostics to therapeutic technologies are assessed rigorously before market, but clinical investigations, essential for initial safety and efficacy, may not fully capture the complexity of real-world use and long-term outcomes and may not be feasible for lower-risk devices.
What real-world data add
Post-market clinical follow-up (PMCF) includes the collection of clinical data from real-world use, to further evaluate a device when it is used in a broader patient population. Real-world data offer insights into the safety, effectiveness and long-term performance of medical devices used outside the controlled conditions of clinical investigations, across diverse patient populations and clinical settings. Unlike prospective interventional studies and randomised controlled trials, real-world data reflect the broader population, including patients with various comorbidities, varying degrees of disease severity and other factors that may not be well represented in traditional clinical trials. The use of real-world data provides a more accurate and comprehensive understanding of how devices perform in real-world settings, helping to identify issues that may not have been detected during pre-market evaluation. By leveraging real-world data to complement traditional clinical trials, researchers and regulators are better equipped to address gaps in evidence, improve post-market surveillance and guide device development in a more patient-centred manner.
Data from standard clinical investigations and data from real-world evidence compare as follows.
| Data from standard clinical investigations | Data from real-world evidence |
|---|---|
| Carefully selected inclusion and exclusion criteria | Not limited to a selected patient group |
| Intended to address a certain hypothesis or clinical question | To confirm the performance and safety in a routine clinical setting |
| Limited to a predefined sample size and time period | Not necessarily limited to a sample size, and suitable to collect data over the device lifetime |
The integration of real-world data into PMCF activities allows for the continuous monitoring of the safety and performance of medical devices. By analysing data from a range of real-world sources, manufacturers can identify emerging risks, assess long-term performance and make necessary adjustments to their products or to their post-market surveillance and PMCF plans. Real-world data give regulators a more comprehensive understanding of a device's performance across diverse patient populations and clinical settings. Real-world data significantly enhance PMCF by providing timely, relevant insights from diverse patient populations and clinical settings. Whether through registries, retrospective studies or user surveys, among other sources, real-world data enable manufacturers to close clinical evidence gaps, refine risk management and maintain regulatory compliance. As healthcare systems and data infrastructures evolve, robust PMCF strategies grounded in real-world evidence will be essential for ensuring device safety and public health.
Gaps a clinical investigation leaves
Real-world data can help bridge gaps in clinical investigation data, particularly for devices used in rare conditions with unique patient populations, ensuring that regulatory decisions are based on the best available evidence. In one team's practice, real-world data typically support very narrow indications, such as distal femur fracture with intra-articular extension. Another important use of real-world data is to provide paediatric data. In both cases, running a traditional clinical investigation to collect the data would be very time-consuming, with long enrolment at few sites and hurdles for the approval of paediatric studies, and expensive.
Sources of real-world data
Numerous sources of real-world data can support PMCF activities, and the most widely used include the following.
- Electronic health records provide large real-world data sets with information about diagnoses, treatment plans and results, and might also include device brand names, catalogue numbers or the unique device identifier. One team also uses these data in combination with insurance claims databases. More data vendors are also providing services to analyse unstructured data, such as clinical notes and imaging, to enable more sophisticated analysis of device performance outcomes. Manufacturers should be aware of potentially limited data quality, and of bias due to the design or analysis of electronic health records.
- Retrospective reviews of medical records are a good source of real-world performance and safety data, and unlike surveys they do not rely on user compliance. Although they require a predefined study protocol and statistical analysis plan, they are more cost-effective than prospective studies and have a lower selection bias.
- Registries are good sources of real-world evidence, with the potential to collect long-term data. So far, national registries are limited to certain devices, such as orthopaedic implants. As an alternative, manufacturers can initiate their own medical device registries, which usually follow a predefined study protocol that can be targeted to the collection of the most relevant performance and safety parameters.
- Laboratory information systems store and manage data on laboratory tests, including blood work, diagnostic imaging and other tests that inform patient care. They can provide important real-world data on how devices perform in relation to specific diagnostic or therapeutic procedures. For instance, devices such as in-vitro diagnostic tests or point-of-care testing devices can be tracked through these systems to assess their accuracy, reliability and potential for misdiagnosis in clinical practice.
- Surveys can collect data from healthcare professionals or patients, and can be designed to collect general feedback on user experience and user satisfaction or to collect data on specific procedures and device usages. Despite their known limitations, such as low response rates, they reach various users in a relatively short time.
- Social media listening identifies early signals of safety issues, user concerns and real-world device performance in general. By monitoring public posts, reviews and discussions, manufacturers can detect adverse events, misuse or unmet needs not captured through traditional channels. Analysing this user-generated content provides timely insights, but it can be difficult to weigh against other sources of data that are monitored by healthcare professionals.
Cases of closing an evidence gap
A national joint registry
A manufacturer of orthopaedic implants, which are considered medium- to high-risk devices, wanted to use publicly available joint prosthesis registries to retrieve performance and safety data for its device. National joint registries are valuable sources of real-world evidence, especially for orthopaedic implants, because of their long-term follow-up data. However, these registries typically do not provide device-specific data in their standard annual reports, which limits manufacturers' ability to assess and compare the performance of individual devices. To address this, the manufacturer requested two additional device-specific reports from the registry owner, one on its own device and another on a group of benchmark devices. These reports enabled direct comparison with the state of the art and will now be received annually, which enhances the value of the registry as a continuous source of real-world evidence. Smaller registries may not have the resources to generate custom reports for manufacturers, or may not agree to provide data for comparator products.
A retrospective review of hospital records
A manufacturer of vascular stents, considered high-risk devices, had strong clinical evidence supporting the device's use in lower-leg arteries, in line with part of its intended use. The device's broad indication, which included use in upper-thigh arteries, lacked robust clinical evidence and relied only on isolated case reports. To address this gap, the manufacturer identified a hospital that frequently used the stent for upper-thigh lesions and conducted a retrospective analysis of the hospital's database, successfully gathering performance and safety data to support the broader indication. Such a review is not typically a continuous activity, and a single centre may not have sufficient volume to provide enough cases for the specific indication.
A user survey for accessory devices
A manufacturer offered low-risk devices primarily used as accessories in interventional procedures, which were not typically featured in scientific publications. Given their accessory role and simplicity, it was neither practical nor necessary to conduct clinical studies, yet the manufacturer still needed performance and safety data to support the device's use. The manufacturer implemented a user survey with a simple case report form, completed during or immediately after an intervention with the device, which enabled the collection of relevant data on technical performance and potential safety events. Short-term follow-up was sufficient because of the device's nature and intended use. User surveys for real-world evidence in PMCF may face challenges with response bias, limited clinical depth and inconsistent data quality, potentially undermining the reliability of safety and performance insights.
Social media listening for a wearable monitor
A manufacturer of wearable cardiac monitors aimed to enhance post-market surveillance by exploring non-traditional data sources. Despite formal reporting channels, some users shared device issues, such as skin irritation or inaccurate readings, only through social media platforms, and conventional post-market surveillance systems missed these signals. The manufacturer implemented a social media listening tool to monitor public posts related to its product, which enabled early identification of recurring user complaints and prompted further analysis of the published literature, complaint and incident databases and other sources. The approach improved patient safety and supplemented traditional post-market surveillance data. It may be more cost-effective for companies to embed social media listening in an overall vigilance strategy than to use it for the PMCF needs of a single product.
Real-world evidence under the EU regulations
A 2024 review of the European device regulations highlights the opportunities they give manufacturers of legacy devices to run real-world evidence studies that bridge gaps in clinical evidence. The review holds that real-world data used in PMCF studies and clinical evaluation reports can help manufacturers adhere to the European Medical Device Regulations. It holds that the primary value of real-world evidence lies in giving an accurate, and therefore more reliable, measure of device safety and performance, because real-world data often mirror routine cases and provide more data than the clinical studies run for a device's market approval. In its account, real-world evidence supplements the clinical evidence from pre- and post-market clinical investigations, reducing the costs of those studies and supporting the manufacturer's benefit–risk conclusion. The review proposes an initiative in the EU similar to the National Evaluation System for health Technology (NEST) partnership sponsored by the US Food and Drug Administration (FDA).
Writers need to be proactive as device frameworks change and may come to use real-world evidence more extensively, possibly through an initiative like NEST in regions outside the US. They should understand how real-world evidence can support the clinical evidence the Medical Devices Regulation requires, reduce costs and contribute to ongoing benefit–risk evaluations, particularly for legacy devices, and know that device real-world evidence comes from many sources, from hospital charts to social media listening.
The US framework for device real-world evidence
The FDA's Center for Devices and Radiological Health oversees device approvals, which have evolved from the Medical Device Amendments of 1976 to more recent policies such as the 21st Century Cures Act of 2016 and the Food and Drug Omnibus Reform Act of 2022. A 2024 review of the US framework reports that since 2017 the FDA has increasingly recognised real-world data and evidence as part of the evidence package for device approval, particularly for post-market surveillance, breakthrough devices and alternative approval pathways. It names data availability, reliability, harmonisation and interoperability as remaining barriers, and describes NEST as a database established to synthesise real-world evidence from clinical registries, electronic health records and claims data. It expects the regulation of device real-world evidence to focus on improving data transparency, standardisation and analytical methods to enhance regulatory confidence, with advances in AI and machine learning helping to analyse large-scale real-world data. It holds that expanding real-world evidence beyond safety and efficacy to include cost-effectiveness and patient-centred outcomes will further support its role in regulatory decision-making and health technology assessment. It warns that, although regulatory acceptance of real-world evidence is growing, disparities in the affordability of treatments, together with the related reimbursement policies, may compromise patient access to innovative devices.
Writers need to be aware of the value of integrating real-world evidence into device submission dossiers and of the regulatory framework for doing so. By applying the evolving guidelines and best practices for using real-world evidence, writers can support teams in producing applications that draw more robust conclusions about the need for the device at hand.
The Open Hand Initiative for in-vitro diagnostics
Real-world data are increasingly used to support regulatory decisions, but integrating them into in-vitro diagnostic approvals remains complex because of issues of data quality, availability and regulatory clarity. The Open Hand Initiative, a collaboration of the FDA, device manufacturers and the Medical Device Innovation Consortium, was piloted to address these challenges in moving SARS-CoV-2 serology tests from emergency use authorisation to full market approval. It promotes transparency by encouraging manufacturers to share insights from their regulatory interactions to improve the quality and applicability of real-world evidence. Participants identified difficulties with data access, standardisation and interoperability and limits on using retrospective data in regulatory submissions, and the COVID-19 pandemic added barriers of its own: shifting diagnostic criteria, evolving patient populations and rapid advances in test development. The initiative emphasised the need for clear regulatory expectations for real-world evidence, improved data collection and validation, and better alignment of study design with FDA requirements, and it highlighted data privacy compliance and use of existing real-world data sources as critical for successful submissions. It showed the potential of regulatory science in which industry and regulators refine real-world evidence standards together, and although it began with in-vitro diagnostics, the model could be extended to other devices and to medicines.
Writers involved in regulatory documentation, clinical investigation design and evidence synthesis for FDA submissions can benefit from understanding how to align real-world data collection with regulators' expectations, to improve the quality of the data and the transparency about the types of data supporting a submission. Addressing regulators' expectations in the content of the documentation to be submitted may shorten the authorities' time to respond and lead to a more successful submission.
Statistical methods for setting endpoints from real-world data
A statistical method uses real-world evidence to propose meaningful endpoints for assessing devices. It applies entropy balancing, a data processing method that matches treatment and control observations, to address possible differences between a study's target patient population and existing real-world patients, taking account of operational differences between clinical studies and real-world practice. Applying it reduces the risk of drawing biased conclusions from a set of different sources, as is often the case with real-world data.
Writers need not be responsible for the statistical methods, but they should understand how real-world evidence can help select relevant performance and safety endpoints for device studies and assessments. These parameters are used at several steps of a device's clinical evaluation, such as defining the state of the art and designing the endpoints of a clinical investigation, and they are hard to set when reliable, relevant data are scarce. Writers need to be careful when drawing conclusions on a device's safety and performance from unstructured data, because the risk of a biased conclusion is high. Weighting different formats and sources of data is not easy, but statistical methods like entropy balancing can play an important role in producing a robust body of data from which conclusions can be drawn safely, and writers should be aware of these methods and work with statisticians to analyse and assess data. Adopting robust methods can ensure more accurate clinical evaluations and support the validity of regulatory documentation.
AI-driven meta-analysis of clinical data
Primary registries such as ClinicalTrials.gov, which hosted over 300,000 clinical trials in 2020, and the European database on medical devices (EUDAMED), then still to come, together with the real-world data private registries collect, supply manufacturers with large datasets from which to draw insights for market analysis and their business. Health apps, wearables, sensors and software produced through partnerships such as Zimmer Biomet with Apple and Novartis with Microsoft increasingly let patients report the effects of a device or treatment in near real time.
Publicly available clinical data are not necessarily actionable. Reading the information recorded in a clinical trial through meta-analysis takes industry expertise. Done manually, the process can be cumbersome and does not usually allow insights beyond the regulatory requirements of a clinical evaluation. As of 2020, clinical data were recorded in different ways depending on the study format and database, their structure and quality still needed to improve, and initiatives such as EUDAMED were helping to structure them. A developer of such algorithms holds that network meta-analysis powered by artificial intelligence can evaluate a manufacturer's clinical data to see whether a study is likely to meet the expectations of health authorities, show how a clinical trial or product could be improved, point to a pathway to funding and reimbursement, a new market or a gap in the market, and show where to allocate resources.
Keeping device documentation current
The use of real-world evidence in the device sector continues to expand, and it offers opportunities to strengthen clinical evidence, enhance post-market surveillance and support regulatory submissions. Writers must stay informed of emerging methods, evolving regulatory frameworks and collaborative initiatives such as the Open Hand Initiative, so that device documentation is robust, relevant and aligned with current standards.