Choosing a PMCF user survey and its evidence level
A user survey is a popular route to PMCF data, and two different instruments share the name and sit at different levels of the hierarchy of clinical evidence. Which one suits a device turns on the device's risk, on how much clinical evidence already exists for it and on what a respondent can actually report.
What a user survey offers
A survey is quicker and cheaper than a full clinical investigation and still produces real-world evidence: nobody is asked to use the device, so what comes back is how it is already being used in practice. It is an accepted method, it can reach users virtually across the globe, and many companies have experience of helping to implement them.
A survey can serve several purposes. Its cost-effectiveness makes it suitable as the recurring collection that keeps PMCF data and the clinical documentation current. It can be a good way of confirming the safety and performance of a legacy device being moved from the directive to the MDR. It can help fill a clinical gap, such as a patient population with little data behind it. It can supply complementary data for a new model of an existing device in place of a full clinical investigation on that model. It can potentially supply data for a device's market submission. A survey can also test marketing or other claims the manufacturer wants tested, and help support them.
A survey in the hierarchy of clinical evidence
Clinical evidence for a device is ranked in a hierarchy of twelve levels, from the results of high-quality clinical investigations at level one down to preclinical data and bench testing at level twelve. A survey can sit at either of two of those levels. Level eight is defined as proactive post-market surveillance data such as that derived from surveys, and the general end-user survey falls there. Level four is defined as outcomes from studies with potential methodological flaws whose data can still be quantified and justified. A survey can fall at level four as long as statistical analysis and a rationale stand behind its planning and the data collected can be analysed, quantified and justified.
General end-user and case-specific surveys
A level eight survey and a level four survey are different instruments. A level eight survey screens users on their usage across a set period, such as the last month or the last twelve months, and each returned survey covers that respondent's multiple uses of the device within it. The respondent completes it once, from recollection, and the design is solely retrospective. It is quicker and cheaper to implement, but its evidence is weaker, mainly because of that recollection, and its analysis is less specific. It is usually used for lower-risk devices, or for well-established legacy devices that usually have sufficient clinical data behind them and a smaller clinical gap to fill.
A level four survey is usually used for higher-risk or implantable devices, or for a device with little clinical data already collected and so a larger clinical gap. It screens users on their previous and potentially upcoming usage, and each returned survey stands for one patient or one case. A respondent who used the device several times can therefore complete several surveys, and the collection can mix retrospective with prospective use. Its evidence is more reliable and more specific: an adverse event reported in a case-specific survey can be tied back to one use of the device, which a general survey is not granular enough to do. In return, its data collection is usually more complex, may run over a longer period and is likely to cost more.
Across industry the level four instrument tends to be called the high-quality survey, although a level eight survey that can be justified can also be designed to high quality. One survey provider ran only general end-user surveys in its first two and a half years of PMCF surveys, and then saw notified-body guidance pushing towards level four surveys, from about eighteen months to two years before February 2024.
The two forms can also be combined to fit what can be collected. For a legacy implantable device that is also a well-established technology, such as a bone screw, lower-level data collection can probably serve, provided the planning of the level eight survey is very thorough and still sets endpoints with acceptance criteria or their justification. The survey design may then stay close to a general end-user survey, while the planning and the analysis against the endpoints follow a level four survey. For such a device, the per-use detail a level four survey asks for may not be recorded in the patient chart, and a respondent may not easily answer on an individual usage. There has to be some balance between what can actually be collected and the way it needs to be collected, while the statistical planning behind it, with its endpoints and criteria, stays in place.
A survey or a clinical study
The line between a PMCF survey and a PMCF clinical study is a tricky grey area, on which standpoints differ. A level four survey falls closer to a clinical investigation, but its collection is still retrospective, with the respondent reporting on use of the device that has already happened in the real world. It stays focused on the defined endpoints, and detail such as device sizes or patient population is collected only to give the data granularity. And it does not seek patient-identifiable information.
The choice between the two depends on how much evidence already exists. A very large clinical gap, or a device with no clinical data collected at all, probably points towards a clinical study or investigation first, while for legacy devices surveys are a very good option. It also depends on what a respondent can actually supply. Implantable devices may be appropriate for a level four survey, but very often they are not. Information on a hip implant has to be collected across the device lifetime, but the physician may be able to report only on the procedure: the patient may not return to the same physician, and may not return at all if everything is fine. So a survey is hard to make a relevant option for devices like that.
Difficulties of user surveys
User surveys are a popular route to PMCF data and a good option for devices used as accessories and for lower-classification devices such as class I and class IIa. The first difficulty is identifying the users of the subject device so that the survey can be addressed to them, which is hard for a device such as a guide wire that is used in many different institutions. The devices that surveys usually cover are also not of great scientific interest.
So convincing physicians to complete a survey can take a lot of work. A manufacturer may set the survey up, comply with data protection rules and distribute it to many patients and physicians only to receive a low response, and the returns from physicians are sometimes incomplete, with data quality that may be rather low.