Post-authorisation safety studies and their reports

A post-authorisation safety study is a study of an authorised medicine that a regulator can require to settle a question about its safety. Most are non-interventional. Such a study is registered, its final report is written to a mandated format that differs from the clinical study report of an interventional trial, and a study a regulator requested is submitted in common technical document format, which leads sponsors to compare it with their pivotal trials.

What the study is and why a regulator can require one

A post-authorisation safety study is any study relating to an authorised medicinal product conducted with the aim of identifying, characterising or quantifying a safety hazard, confirming the safety profile of the medicinal product, or measuring the effectiveness of risk management measures, as Article 1(15) of Directive 2001/83/EC defines it. ^{1, 2, 3} In particular, these studies are conducted to quantify potential or identified risks, to fill gaps in existing safety data, to define risks, or their absence, further — after long-term use, for example — or to assess the effectiveness of a risk minimisation activity, and as such they may form part of a risk management plan. ^{1, 2} The concept predates the Pharmacovigilance Risk Assessment Committee by more than a decade, but most regulatory writers would have been unaware of these studies until the changes in European regulations that mention them as part of a general strengthening of pharmacovigilance procedures. ^{1, 4}

Randomised clinical trials are considered to sit atop the hierarchy of clinical evidence and form the basis for most drug approvals, while non-interventional and observational studies are considered a weaker form of evidence and had, until recently, received little attention from regulatory agencies. ² A clinical development programme, however exhaustive, will always be subject to certain limitations: the number of patients exposed may be insufficient to detect a small but significant safety signal, and the controlled setting of clinical trials may not adequately reflect clinical practice, in which real patients may, for example, be multi-medicated or have more concurrent illnesses. ^{1, 2} The number of patients exposed or the duration of exposure in a development programme may also not be sufficient to detect a rare but important safety signal. ²

These studies are one of the tools available to the Pharmacovigilance Risk Assessment Committee for monitoring approved drugs, and an important one for assessing how a medicine behaves outside the confines of clinical trials. ^{1, 2} Entities such as the committee can use them to oblige drug companies to collect data on approved products. ⁵ A post-authorisation safety study is often required as a post-marketing commitment to establish a safety profile or to address specific safety concerns. ⁶ The non-interventional post-authorisation safety study is an increasingly common pharmacovigilance measure, carried out after a medicine has been authorised to obtain further information on its safety. ⁷ While observational studies have a long pedigree, the value of conducting these studies has gained increasing regulatory attention, and the European Medicines Agency has published a template, similar to that for clinical trials, to aid harmonisation of their reporting. ⁷

Interventional and non-interventional studies

A post-authorisation safety study may be a clinical trial or a non-interventional study. ^{3, 1, 2} An interventional study is, by and large, conducted and reported in accordance with the familiar International Conference on Harmonisation guidance. ^{1, 2} If the study is a clinical trial, Directive 2001/20/EC details the legal obligations for good clinical practice in its conduct. ³ In a non-interventional study, treatment is assigned by decisions guided entirely by clinical practice and administered according to the approved labelling, with no additional protocol-mandated procedures or tests. ^{1, 2} A non-interventional study should meet the requirements of Volume 10 of the rules governing medicinal products in the European Union, and its purpose is to generate data of clinical or public health importance. ³ The majority of these studies are non-interventional. ^{2, 5} Non-interventional studies can include literature reviews or retrospective analyses of registry data as well as observational studies, but observational studies are the most common. ^{1, 2} They are also the ones regulatory writers will most likely encounter. ¹ A non-interventional study is still conducted largely in the general spirit of the International Conference on Harmonisation guidance and of good clinical practice, but certain aspects may differ. ^{1, 2} In particular, where submission to the committee is required, its final study report should be based on the guidance issued by the European Medicines Agency and differs in many features from a typical clinical study report for an interventional trial. ^{1, 2}

Protocol and final study report

Companies are required to provide a written study protocol before the study begins, with the format and content set out in Module VIII of the good pharmacovigilance practice guideline. ³ Pharmacovigilance data and any new information generated should be monitored and the benefit-risk balance considered, and information from the study should be included in Periodic Safety Update Report and risk management plan updates. ³ The final study report should be submitted as soon as possible after finalisation and within twelve months of the end of data collection, and it should include a publicly available abstract. ³ For the writer, this report is where pre-authorisation writing meets post-authorisation writing, because the skills it needs are similar to those needed for a pre-authorisation clinical study report. ³

Registration and posting of results

Methodological details of every post-authorisation safety study should be posted to the EU PAS register, run by the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance. ^{8, 9} Much has been made of the need to disclose interventional trial protocols and results in the interests of transparency, and this is the analogous requirement for non-interventional studies, presumably a further effort to increase transparency and strengthen pharmacovigilance procedures. ^{8, 9} The results should also be posted within two weeks of submission of the final study report, which is usually submitted within a year of the end of data collection; some companies post the entire report, redacted and stripped of appendices, and others post only the report abstract. ^{8, 9}

Most companies now have groups dedicated to posting details of interventional trials on sites such as ClinicalTrials.gov, but observational studies, particularly older ones, may have been overlooked. ^{8, 9} So it is worth checking early in the drafting that the study is registered and has an EU PAS registration number, because the final study report cannot be completed without it. ^{8, 9} The number is also required on the cover page, whose format the guidance mandates. ^{10, 11}

The structure of the final study report

Guidance covering the format and content of the final study report of a non-interventional post-authorisation safety study was issued in 2013, and it suggests that its own table of contents can be used to build the report template. ^{12, 13} Where that contents list is not used directly, it would still seem advisable to stay as close as possible to the structure it gives, and a sensible approach might be to keep its headings, marked not applicable if appropriate, and add extra headings and subheadings where necessary. ^{12, 13} By analogy with reports written to ICH E3, the sections the guidance covers do not have to become separately numbered sections, so the abstract need not be numbered as section 1. ^{12, 13} The type of study can vary widely, a single template might not always cover the reporting needs, and often there will be headings without any content. ¹³

The structure the guidance sets out runs as follows.

1. Abstract
2. List of abbreviations
3. Investigators
4. Other responsible parties
5. Milestones
6. Rationale and background
7. Research question and objectives
8. Amendments and updates
9. Research methods
   • 9.1. Study design
   • 9.2. Setting
   • 9.3. Subjects
   • 9.4. Variables
   • 9.5. Data sources and measurement
   • 9.6. Bias
   • 9.7. Study size
   • 9.8. Data transformation
   • 9.9. Statistical methods
     • 9.9.1. Main summary measures
     • 9.9.2. Main statistical methods
     • 9.9.3. Missing values
     • 9.9.4. Sensitivity analyses
     • 9.9.5. Amendments to the statistical analysis plan
   • 9.10. Quality control
10. Results
    • 10.1. Participants
    • 10.2. Descriptive data
    • 10.3. Outcome data
    • 10.4. Main results
    • 10.5. Other analyses
    • 10.6. Adverse events/adverse reactions
11. Discussion
    • 11.1. Key results
    • 11.2. Limitations
    • 11.3. Interpretation
    • 11.4. Generalisability

^{12, 14}

The revised European Union pharmacovigilance legislation implemented in July 2012 resulted in new agency guidelines for post-authorisation safety studies covering protocols and study reports. ¹⁵ One project then set out to develop enhanced protocol and report templates carrying additional language that reflects the study designs typically used to evaluate safety and other outcomes in real-world settings. ¹⁵ Candidate text was drawn from the published literature and from design-specific documents, including regulatory and manuscript guidelines for interventional trials and for non-interventional studies, and each selected passage was associated with its study designs and marked as instructional, required or optional. ¹⁶ The project reports that aligning the guidance with design-specific text improved template quality for regulatory submissions, and that the publication-guideline text in the report templates let authors identify important content when writing peer-reviewed publications. ¹⁷

The abstract

Where a report written to ICH E3 opens with a synopsis, this report has a structured abstract, similar in some ways to a journal abstract but with more subheadings. ^{18, 14} Besides the title and key words it covers rationale and background, research question and objectives, study design, setting, subjects and study size, variables and data sources, results, and discussion, and the guidance states that the word count, excluding the title and certain administrative details, should not exceed 500 words. ^{18, 14} With so many subheadings, and for a study of any complexity, that will be challenging, and as far as one experienced writer is aware the count can be exceeded, just as an ICH-based synopsis should not exceed three pages but has some flexibility. ^{18, 19} Sensible advice is to keep as close to 500 words as possible without omitting any important feature, result or conclusion of the study, particularly where the abstract rather than the entire report will be used to disclose the results. ^{18, 19}

Research methods

As in a report written to ICH E3, the first part covers administrative matters — investigators, other responsible parties, milestones — and then the research methods. ^{20, 21} Where the protocol itself was written to the latest protocol guidance for these studies, the methodology sections can be adapted from it, but the correspondence is not exact: report subsections such as bias, subjects and sensitivity analyses have no exact protocol counterpart, although bias and sensitivity analysis may have been addressed in protocol sections on data analysis and on the limitations of the research methods. ^{20, 21} When the protocol is written, it might help to have the final-study-report guidance to hand as well as the protocol guidance, to ease the later drafting of the report. ^{20, 19}

The methods section of this report should perhaps be more stand-alone than its ICH-based equivalent. ^{20, 21} Extensive cross-referencing to the protocol is generally considered good practice in a clinical study report, but here the protocol may not necessarily be appended to the report, or may be redacted during disclosure. ^{20, 21} The details of the methods should in any case be available on the register's website, and a more stand-alone methods section enhances readability. ²¹ A study started before the committee became operational in 2012 is unlikely to have been conducted with a protocol drafted to the latest guidance. ^{20, 19} In some cases a study intended largely as a marketing exercise may have been designated a post-authorisation safety study afterwards. ²⁰ The original protocol of such a study may not resemble the mandated protocol format, and the methods section then requires more thought and more work. ^{20, 19} Such a study will not have been submitted to the committee either, and the writer will have to refer to the guidance text to ensure that the content is appropriate, especially as some sections will be unfamiliar to someone used to writing reports to ICH E3. ^{20, 19} Unlike its equivalent in an ICH-based report, the section headed amendments and updates refers only to protocol amendments, and changes to the statistical analysis are presented as part of the results. ^{20, 21}

Results

The results section has six subsections. ^{22, 23} The participants subsection is self-explanatory. ^{22, 23} Descriptive data refers largely to patient characteristics, and because these studies are by definition non-randomised, a good understanding of the baseline characteristics of the different patient groups is important for assessing potential biases when the groups are compared. ^{22, 23} Outcome data should include, by the guidance's brief text, the numbers of subjects across categories of main outcomes, and some of the few available reports have read that as outcomes such as pregnancies. ^{22, 23} But the phrase numbers of subjects perhaps suggests that patient results as such do not belong here, despite what the heading implies. ²² Outcome results can go in other sections, such as main results and other analyses, so another reading is that the subsection could serve as roughly the equivalent of the analysis populations section of a report written to ICH E3. ^{22, 23} The subsection is likely intended to reflect the considerable amounts of missing data observational studies often carry. ²³

The last subsection, adverse events and adverse reactions, has detailed guidance of its own and will likely closely resemble the adverse-event reporting of an ICH-based report. ^{22, 23} Where applicable, a clear, well-structured subsection lets the data be taken readily into other documents, such as a Periodic Safety Update Report or a Periodic Benefit Risk Evaluation Report. ^{22, 23}

Discussion

The standard advice for many clinical study reports is to keep the discussion brief and fairly non-committal, on the argument that higher-level documents such as the clinical overview are the more appropriate places to relate a study's findings to the rest of the development programme and to the literature. ^{24, 25} A post-authorisation safety study report, however, does not form part of a development programme and so may be more likely to be read on its own. ²⁴ Its discussion is structured into key results, limitations, interpretation and generalisability, four subsections that invite an involved discussion and make one more difficult to avoid. ^{24, 25}

As with the results, the discussion will be easier to write where the protocol was written to the template. ^{24, 25} The limitations subsection can largely be based on the protocol's limitations of the research methods, embellished with what the results have since made clear. ^{24, 25} Most observational studies will be subject to similar limitations, bias among them, and similar strengths, such as greater applicability to clinical practice, which the generalisability subsection addresses directly. ^{24, 25}

Appendices and annexes

The template offers appendices, which would likely hold key study documentation such as the protocol and summary tables kept out of the report body; no structure is given for them, so following the company's approach for reports written to ICH E3 is probably reasonable. ^{26, 27} Annex 1 is mandatory and lists the documents available on request, such as listings, and Annex 2 takes any additional information. ^{26, 27}

Submission beside the pivotal trials

When a regulator requests one of these studies, regulatory submission in the usual common technical document format is expected, and an imposed study must be submitted in that format. ^{7, 6} This inevitably leads sponsors to wish to compare their pivotal clinical trials with the study. ⁷ A study of this kind is requested for about half of new substances, and its scale — usually a much larger enrolled population than a trial's — gives it particular value for identifying rare adverse events and for providing reassurance about established safety knowledge. ⁷ Its observational data, usually limited but of high external validity, can complement the clinical data, usually thorough but of lower external validity. ⁷ Comparability is limited by differences in design and conduct, among them selection bias, which is harder to control in the observational setting, and the typically higher extent and quality of data collection in the clinical setting. ⁶

What can be collected

The observational setting limits the data that can be generated, chiefly because interventions other than those that would occur during routine treatment or clinical practice are not permitted. ²⁸ That rules out testing of any kind — laboratory tests, X-rays, vital signs — as well as more intensive questioning or study visits beyond routine care; where the product label suggests additional monitoring, data can and should be collected as indicated. ²⁸ Treatment is solely at the investigator's discretion, so the freedom of dosing, of duration, of stopping and starting, of dose changes and even of changes of treatment can confound interpretation within the study while providing useful real-world information. ²⁸

Missing data

Extraction from medical charts is permissible, but monitoring is likely to be less intensive than in a trial, which makes clarifying missing data challenging, and the duration of many such studies may make clarification at a distance in time difficult or impossible. ²⁸ Even basic data such as patient age, sex and disease history, let alone more critical information such as adverse events or causes of death, are far more likely to be absent than in a trial. ²⁸ The extent of missing data must be considered in any comparison with prior trials, the number of missing data points should be quantified wherever possible, and imputation methods must be described in detail, along with any sensitivity analyses. ²⁸

Endpoints

Endpoints requiring measurement or patient questioning are likely to take place less often in the observational setting than in trials, which limits the value of comparisons, and the extent and reliability of data collection are usually lower. ²⁸ If adverse events are recorded systematically, typical differences from a trial include longer intervals between patient contacts, a longer study that increases reporting fatigue and the risk of loss to follow-up, and a focus on particular or established rather than unexpected safety issues, and details such as start and stop dates, severity or countermeasures are more likely to be vague or missing entirely than in a closely monitored trial. ²⁸ For most soft endpoints such as biomarkers or quality-of-life measurements, and for those at high risk of reporting bias such as patient-reported outcomes including adverse events, comparability with a trial is often limited, while harder endpoints such as survival may be more reliable. ²⁸ Where an overt comparison of adverse event rates between the two settings is included in Module 2.7.4, regulators are well aware of these systematic effects: a lower incidence rate in the observational study may not be very informative, while a notably higher one will probably need explaining, as would a higher rate in the observational study for an adverse-event primary endpoint that is a single event or class of event. ²⁸ Some outcomes recorded may lend themselves well to comparison with the earlier phase III studies, in particular analyses based on spontaneously reportable events at low risk of being missed or falsely recorded. ²⁹

Demographics and baseline characteristics

Demographics and baseline characteristics take on a much more important role in this submission than in a typical clinical submission, which can often be summed up as the treatment groups having been well balanced. ²⁸ The reason is that the selection of patients and of treatments by investigators, which would render a clinical trial useless, is one of the most important pieces of real-world data to emerge from such a study, especially where the screening failure rate is low. ^{29, 28} The real-world population usually differs from the trial population in a number of ways, because trial inclusion and exclusion criteria tend to select patients who are exemplary for the target indication but lack severe comorbidities. ²⁸ Where the indication is quantified or graded, the range of severity in the observational study is likely to be wider, including both sicker and less sick patients, and trial selection tends to reduce the proportion of patients with other diseases, while the observational study should have no such restriction beyond those in the label. ²⁸ That produces a wide variety of confounding factors whose impact on the main safety results has to be considered; differences between groups in a multiple-arm study should be discussed with the possible sources of bias named; and the extent of comorbidity and disease severity are worth considering as subgroup definitions no later than the drafting of the statistical analysis plan. ²⁸ A post-authorisation safety study is by definition preceded by a phase III submission, and the phase III studies typically inform its design. ³⁰ Its inclusion and exclusion criteria should be minimal, usually broad enough to capture every patient who receives the treatment at the study sites, although in some cases, particularly in a multiple-arm study, some effort will be made to recruit similar patients across arms, or at least to restrict the study to the indication. ^{30, 29}

Subgroups

In a larger study, subgroup data may take more prominence than in a submission built on pivotal trials. ²⁸ Such studies are almost never powered for subgroups and formal conclusions cannot be drawn from them, but the number of patients can provide particularly strong reassurance, or evidence of a higher adverse event rate in a particular group. ²⁸ Because of their large scale, these studies offer the opportunity to assess safety across multiple subgroups with greater certainty than is possible in a clinical trial. ⁶

Terminology

Effectiveness is preferred over efficacy for the results of an observational study, and when data are compared directly it could be useful to reserve study and patient for the observational work against trial and subject for the clinical work, although such distinctions will not excuse an otherwise inadequate distinction between the data sources, and imposing them generates additional writing and quality-control effort. ^{31, 28} Hard endpoints such as death, recurrence of the disease under treatment or hospitalisation can often be evaluated from routine data collection without prejudicing the observational status of the study, and where they are comparable to efficacy endpoints from the trials they can be detailed in Module 2.7.3, provided it is made clear that formally the results arise from safety analyses. ²⁹ Because these studies are not conducted to investigate efficacy, no efficacy claim should be made even for an endpoint that lends itself to one and shows effectiveness similar to the clinical setting. ²⁹ Comparisons of effectiveness with clinical efficacy data are subject to the same caveats as all other endpoints, because of the considerable differences in study conduct. ²⁹

Selection bias

Many of the tools that reduce bias in a trial, such as blinding and randomisation, are not available in the observational setting, and potential sources of bias need to be considered very carefully and discussed in detail in any submission of data derived from such a study. ³² These studies are particularly prone to selection bias, and especially to bias in the allocation of patients to treatment groups: one non-interventional study of an anticoagulant showed a clear but unexpected difference in all-cause mortality in favour of the investigational treatment, with the patients receiving it less likely to have cancer at baseline and younger on average than the comparator patients receiving standard treatment. ^{32, 29} Selection bias also applies where an investigator decides whether to include a patient, which can be mitigated by asking investigators to consider for inclusion all consecutive patients being considered for any of the treatments the observational plan permits, reducing the risk that concerns about compliance or likely response to treatment influence the outcome. ²⁹ Prescribing practices for a new medicine change over time, particularly in the first years while experience and knowledge are being gained, and later studies may show different biases from early ones. ²⁹ Even where established imbalances can be traced to particular reasons for clinical decision-making, they should be considered anew with each new study. ²⁹ Selection bias can change with increasing experience of a product, whether because the product becomes established or more or less affordable or because new safety information causes investigators to restrict its use. ²⁸ Clinical investigators also tend to be more experienced and up to date than the medical community in general, and the patients selected and the quality of treatment at a centre of excellence may well be closer to the real world than a trial's and still not be representative of it. ²⁸ The biases inherent in the design of these studies differ considerably from those of clinical trials, which raises challenges for direct comparisons between the two, particularly where the study has more than one treatment group and selection bias may confound comparisons even within it. ³¹ Data quality issues may also complicate interpretation, and direct comparisons with clinical data should be made very cautiously. ³¹

Risks rather than benefits

In contrast to most submissions of clinical data, the focus here is on risks and not benefits, and no new formal benefit information is collected. ^{31, 29, 6} The study nevertheless creates opportunities for interpretation beyond its one or two safety endpoints: the likely real-world usage of the product, how the treated study population differs from the one defined in the prior clinical trials, and how the safety profile compares across a range of subgroups. ^{31, 6} Its submission offers an opportunity to create a robust safety profile for real-world use at a relatively early stage in the product's lifecycle. ³¹

The state of the guidance

When the committee was established, these studies were plucked from relative obscurity and given a much more prominent role, and in 2014 reporting them to the mandated format was still a relatively new undertaking for many companies. ³³ The guidance was also evolving, so it was advisable to check the agency's post-authorisation safety studies pages occasionally for new developments. ³³ Given their novelty, there was no consensus on how these reports should be drafted. ⁴ A consensus was likely to emerge with time, although the wide range of possible study types might slow the process. ³³ For the most part, common sense and analogies drawn from reports written to ICH E3 should be sufficient to produce a report that is fit for purpose. ³³

By September 2017 a writer checking the agency's website could find nothing new of significance for actual report drafting, although detailed procedural guidance had become available. ³⁴ Because some companies include the full report, appropriately redacted, on the network's website, an increasing number of examples were becoming available. ³⁴ The site's search did not allow results to be filtered by whether a final report was available, so they had to be opened one by one. ³⁴ With patience, it should be possible to retrieve some relevant examples of how other writers have approached and interpreted the guidance. ³⁴