The anonymisation report
The anonymisation report accompanies a clinical data publication package under EMA Policy 0070 or Health Canada's public release of clinical information, and explains how the package was de-identified: the anonymisation methodology applied and the reasons for it, how the risk of re-identifying a participant was assessed and managed, and the risk that remains. It follows a template the regulators supply, and published reports show how sponsors have filled each part of it.
What the report is for
An applicant is required by legislation to protect the personal data in the clinical reports it submits, so that neither the trial participants nor the other individuals involved in a study are identified, and the anonymisation report describes how it ensured that protection. The report sets out the anonymisation methodology applied, the rationale for it, and how the risks of re-identification were measured and managed. It should address the risk of re-identification, meaning the probability of re-identifying trial participants once they have been anonymised. Its regulatory purpose is to demonstrate that the changes made in the anonymised documents adequately protect the participants' privacy, to give the rationale for those changes, and to demonstrate that after anonymisation the risk of re-identification on public release is acceptable and the impact on data utility has been considered.
Both the EMA and Health Canada require the report, with the methods used and their justification, to be submitted with the anonymised submission in the proposal package. Under Policy 0070 a sponsor must provide one with each submission, explaining its anonymisation methods and how data utility was maintained. The report is one of two new documents that any application under the policy requires, the other being a table justifying the commercially confidential information.
The report is a requirement the policy introduced. In 2018 regulatory medical writers knew the clinical reports a package discloses well, but the report was unfamiliar to them, and many companies were preparing the redacted package and the report for the first time while the industry gathered experience. Preparation of the report was then in its early stages, and both the EMA and, above all, applicants and marketing authorisation holders faced a steep learning curve before it could be considered a mainstream regulatory document. The report created challenges of its own for sponsors.
The joint template
The anonymisation report template was new with the relaunch of Policy 0070: a downloadable document that the EMA supplies with instructions. It is used for every clinical data publication submission, whether to the EMA, to Health Canada or to both, and in it the applicant describes the anonymisation strategy adopted for each document package and what it took into account in arriving at that strategy. The EMA developed it jointly with Health Canada's public release of clinical information team. Its instructions contain a set of definitions that guide applicants in completing the form for the clinical document package.
Accounts of when the template was released differ. Before the policy's restart, it was expected in good time for September 2023. One account has the EMA and Health Canada releasing it together in October 2023 as a fillable PDF. Another lists the EMA's release of the new form template and its instructions among the regulatory news of May 2024.
The form is built mainly of structured data fields with pull-down menus, and it has a very limited number of free-text sections. A questions-and-answers document published in 2023 accompanies the form, and a new one was expected in 2025 as part of the second step of the relaunch. The form cannot be saved, as the template warns, so it is completed in one go, with all the information prepared beforehand in a format that can be pasted straight into it.
Accounts of the form's sections differ. One gives six, the first two being application information and the anonymisation methodology, where the applicant states which methodology was used. Another gives five, the first being the anonymisation methodology, the second the identification of the data variables and the third the risk assessment, together with a deviations section.
One statistical anonymisation platform generates its own report, holding all the risk results and the degree of transformation applied, such as how many ages were changed. The fillable PDF could not be uploaded into that platform, so its report is downloaded and copied into the PDF. Automated anonymisation methods can generate the associated anonymisation reports automatically, which saves medical writing teams a lot of time and ensures that nothing is left out of the explanatory notes.
The EMA and Health Canada set slightly different requirements in the report for how data are redacted and how the redacted data are labelled. Each agency accepts both conventions, so redaction done to one agency's convention need not be changed before submission to the other.
The Annex 1.2 template
The policy guidance gives the structure and content of the report as a template in its Annex 1.2. Preparing a report under that template takes five steps: choosing the methodology, identifying the direct and quasi identifiers, assessing the anonymisation, considering data utility against the re-identification risk, and a closing declaration.
The first report on the EMA's website to document a quantitative measure of re-identification risk, published in June 2017, was for the orphan drug Alprolix, which is indicated for the rare disease haemophilia B. Biogen, the product's marketing authorisation holder in the United States, prepared it under the Annex 1.2 template, and Swedish Orphan Biovitrum, the holder in Europe, submitted it and revised it after interactions with the EMA. In 2018 the policy, the report included, was still in its implementation phase, and companies and the EMA were still learning, so the holder that submitted the Alprolix report offers it only as an example of how to prepare a quantitative report.
The reports published so far
In 2024 many of the reports on the EMA's website took no standard approach to the risk assessment section, because it is free text, which sponsors approach in slightly different ways.
From 20 October 2016 to 31 December 2017, 64 redacted packages from 29 marketing authorisation holders were published on the EMA's clinical data website, each with an anonymisation report. Twelve of the packages contained no protected personal data. The other 52 reports ran from 4 to 53 pages, and 12 of them were for orphan drug applications and 40 for other applications.
Between October 2016, the first month in which anonymisation reports were available, and 29 March 2018, anonymised clinical reports were published for 77 medicines, among them medicines for use in children. Of the 77, 14 were orphan drugs, 14 generics, two biosimilars and 47 others. Thirteen of the 77 packages were not anonymised, which left 64 anonymised packages. Twelve of the thirteen were not anonymised because only the clinical overview was published, and seven of those were for generics. The thirteenth, for an orphan drug, held no documents.
Choosing the methodology
The first step of the Annex 1.2 template is choosing a quantitative or a qualitative methodology. A qualitative assessment judges the risk of re-identification subjectively, usually as low, medium or high. A quantitative assessment uses data to produce a numerical value that predicts the probability of re-identification. The policy's guidance on the anonymisation process is not intended to mandate any specific methodology. It highlights to applicants and marketing authorisation holders the available techniques and those the EMA considers most relevant, so that the clinical reports submitted for publication are rendered anonymous before publication. The EMA encouraged a quantitative approach while accepting a qualitative one during the pilot phase of the policy's implementation. The Alprolix report used a quantitative methodology, and masking was the technique applied.
Voydeya is a product with orphan drug designation. Its team chose the methodology with data utility in mind, holding that data utility has to be preserved as far as possible. To keep data utility, the team defaulted to the quantitative methodology, which is used where the risk can be quantified with clear numbers or metrics, and used the qualitative one only where the risk could not be quantified. Its report indicated both.
The choice can differ between the studies of one package. The Voydeya package held 25 clinical studies in rare disease, with sample sizes from 8 to 84. Its phase 3 study of 84 patients was assessed quantitatively. For some studies the risk could not easily be quantified because of the parameters the risk depends on, and a phase 2 study of only 10 patients was assessed qualitatively, mainly because of its small sample size. The methodology section of its report lists the clinical studies with the method used for each. Several reports published in 2016 and 2017 described one methodology that appeared to apply to every study in the package. A review of those reports holds that this may be true in some cases but should not be general practice. Because an application usually contains trials of different phases and sample sizes whose re-identification risk may differ, the review suggests that a report be more specific about the methodology for each study.
Of the 45 reports of 2016 and 2017 that assessed the risk of re-identification, 39 assessed it qualitatively, on a scale of low, medium or high. Eleven assessed it quantitatively, by calculating the probability of re-identification, although a summary table of the same reports gives eight. Quantitative assessment became more frequent towards the end of 2017. By May 2018 most published reports had used qualitative methods, semi-quantitative methods had been used more recently, and two reviews of the published reports had shown the use of quantitative methods increasing. One writer held in 2020 that the predominance of the qualitative method fits the use of redaction and masking, and expected it to give way to a predominance of quantitative methodologies. In 2018 the trend was towards quantitative re-identification risk assessment and automated or artificial intelligence systems, as the industry develops new tools and gains experience.
Of the 52 reports of 2016 and 2017, 32 described their anonymisation method as non-analytical, 11 as analytical, and nine did not clearly specify it. The policy does not clearly define either term, and the reports used both rather ambiguously. The terms possibly come from the policy's section 5.4 on the anonymisation process, which says that an applicant may not follow an analytical approach in an initial phase, in which case the risk of re-identification need not be calculated and step 4 of the process can be omitted. Step 4 is the determination of the quantitative threshold of re-identification risk, so it is justifiable that many holders used non-analytical and qualitative as synonyms for their methodology. Several holders described a non-analytical approach as reviewing the documents manually and deciding which text and numbers to redact against predefined criteria, while other reports called an approach that used automated redaction tools non-analytical. All eleven reports that assessed the risk quantitatively also described their anonymisation as analytical. A few reports did not equate qualitative risk assessment with a non-analytical approach, and one described an analytical approach that evaluates the criteria for anonymisation and the expected risk factors on a qualitative basis. The review suggests that holders give clear definitions when they use the terms in a report.
Identifying the data variables
The second step of the Annex 1.2 template identifies the direct and quasi identifiers, whichever methodology is used, and the policy guidance gives examples of both. Under the Annex 1.2 template, a package with neither direct nor quasi identifiers uses a different template, Annex 1.13.
In the Alprolix report the direct identifiers were subject identifiers and study site identifiers. The quasi identifiers were age and birth date, race, ethnicity, country, height, weight and body mass index, serious adverse events, adverse events of interest relevant to haemophilia B or its treatment, medical history, surgery details, bleeding episodes, and calendar dates. Sex and date of death were not treated as quasi identifiers, because all the patients were male and no deaths were reported during the trial.
Some data were treated as extra sensitive in the Alprolix report: HIV status, hepatitis C status and genotype were masked in the clinical reports but left out of the risk assessment. Not all early reports said how sensitive data were anonymised. The policy does not define which data are sensitive, and a review of the early reports recommends that a report define which data it treats as sensitive and how those data were identified and anonymised.
Many trial participants die during a study as a consequence of the underlying disease, yet their data are included in the report datasets. A review of the early reports recommends that a report specify how post-mortem data are dealt with in the anonymisation process.
Whether a risk assessment is needed
The policy gives two options to establish effective anonymisation. The first is to demonstrate that the anonymisation method removes the possibility of singling out an individual patient, of linking records to one, and of inferring information about one. The second is to evaluate the risk of re-identification and demonstrate that the method mitigated it to the lowest level. Policy 0070 highlights the same three components: linkability, singling out or the uniqueness of the data, and inference.
The third step of the Annex 1.2 template assesses how much anonymisation is needed to reduce the risk of re-identification. Where the applicant can confirm or demonstrate that no individual can be singled out, that no records relating to an individual can be linked, and that no information about an individual can be inferred, the risk assessment section is not completed. Where one or more of these criteria is not met, the applicant continues to the risk assessment. The section on fulfilment of the criteria is not needed where a risk assessment is performed.
Of the 52 reports of 2016 and 2017, five took the first option and performed no risk assessment, and 45 took the second and marked the section on fulfilment of the criteria for anonymisation as not applicable. Two reports from one holder used both options.
The risk assessment
The joint template's risk assessment section gives the most context for the anonymisation strategy, since it is where the team explains what it did and how. The template's instructions for it ask for a clear and concise explanation of the selected methodology and why it was used, and of any limitations of the approach. An accompanying guidance sheet sets out the information the regulators are looking for.
The section opens with the reference population, chosen from a pull-down menu, and that choice is an important part of setting the anonymisation strategy. Health Canada recommends that applicants select a reference population for the indirect variables to help estimate the risk of re-identification. The section asks whether the product concerns a rare disease or condition, and a yes opens further fields for its details. A rare disease brings its own implications for the anonymisation strategy and for how the documents are anonymised. Where a special population is involved, the applicant decides what it is, and the form gives examples of what one might be. This information bears on how much anonymisation is needed and which strategies are used. The applicant then enters the initial risk of re-identification: a level from high to low for a qualitative assessment, and a number for a quantitative one.
Under the Annex 1.2 template the risk assessment identifies the possible adversaries and plausible attacks on the data, evaluates the risk of re-identification, sets and justifies a threshold, lists the variables used for the risk calculation, calculates the risk, and de-identifies the data until the risk is below the threshold.
The assessment starts from the worst case: by default the risk of re-identification is taken as 100% where it can be quantified, and as the maximum where it cannot. The EMA recommends setting the risk of re-identification to its maximum, 1. Ideally anonymisation brings the risk down to zero, or to none where it is not quantified, but preserving data utility requires a trade-off, and what remains is the residual risk.
The residual risk is measured against a pre-specified threshold. For a quantitative assessment the policy recommends 0.09 as a conservative threshold, and the EMA leaves the applicant free to choose another as long as it is appropriately justified. One account has a threshold of 0.09 required by both the EMA and Health Canada, with the risk kept as low as possible where it cannot be quantified. Another gives 9% as the acceptable maximum quantitative risk for both agencies. A threshold of 9% is equivalent to a group size of 11, so that a participant grouped with others by similar variables cannot be re-identified from the 10 others in the group.
A qualitative methodology sets no numerical threshold. It uses the arbitrary levels low, medium and high, and the selected level is justified. Example definitions of the levels are high risk for fewer than 100 trial participants in a rare disease, medium risk for 100 to 1,000 trial participants, and low risk for more than 1,000. The level is based on the characteristics of the source data, such as disease prevalence, trial sample size and number of sites. The policy does not detail the scale, and most holders whose reports were published in 2016 and 2017 attempted to define it in their reports.
The risk of re-identifying an individual patient depends on several parameters: the sample size of the study, the characteristics of the data, the potential adversaries, and the likelihood of attacks, and the Voydeya team used software that takes all of them into the risk assessment. The risk takes into account the number of direct and indirect identifiers, the size and nature of the disease studied, such as a rare or common disease or a paediatric population, the number of participants in the study, and the number and distribution of study centres across countries. In the Voydeya package the measure of risk was POSA, a term from the software, expanded as possibility of suspected attack, and the phase 3 study of 84 patients returned 0.077, within the threshold of 0.09. After anonymisation, the risk of re-identification was below 0.09 in all eleven quantitative reports of 2016 and 2017.
The Alprolix risk was calculated on the combined trial population of all the clinical reports in the marketing authorisation application, across a number of scenarios and iterations combining different quasi identifiers, each scenario counting the subjects left unique.
| Redacted quasi identifiers | Unredacted quasi identifiers | Subjects | Number of unique subjects | Proportion of unique subjects (%) |
|---|---|---|---|---|
| Height, weight, BMI | Age, race, country, SAEs, surgeries, bleeding episodes | 167 | 149 | 89.2 |
| SAEs, surgeries, bleeding episodes, height, weight, BMI | Age, race, country | 167 | 124 | 74.3 |
| Age, race, country, height, weight, BMI | SAEs, surgeries, bleeding episodes | 167 | 53 | 31.7 |
| Race, country, SAEs, surgeries, bleeding episodes, height, weight, BMI | Age | 167 | 20 | 12.0 |
| Age, race, country, SAEs, surgeries, bleeding episodes, height, weight, BMI | – | 167 | 0 | 0.0 |
BMI is body mass index and SAE serious adverse event; the number of unique subjects is the number for whom the combination of values in the unredacted identifiers is unique.
In the last scenario no trial participant had a unique value for any of the selected quasi identifiers, and the risk was 0.006.
Data utility
The fourth step of the Annex 1.2 template weighs data utility against the re-identification risk. Because haemophilia B is a rare disease, the Alprolix report masked all the quasi identifiers and the sensitive data at the level of the individual participant, full narratives included, to protect the participants' confidentiality, even though this reduced data utility. The holder still considered the remaining data informative, because the aggregate summaries and analyses carry the most scientific value and remained largely unmodified. The EMA accepted this, although it does not accept the approach by default.
The joint template asks for the variables with the highest data utility, up to five of them. The Voydeya report listed four: gender, age, adverse events and medical history, the data points the team held to be of the highest utility and wanted to retain.
Deviations, the declaration and the comparison with the original
The joint template's deviations section records exceptions in the package and is a minor part of the report. The fifth and final step of the Annex 1.2 template is a declaration that "the anonymisation report has been prepared following the guidance made available by EMA, and the anonymisation techniques have been applied consistently in the preparation of the documents comprising the Final Redacted Document package". Regulators review the applicant's stated rules together with the anonymised clinical documents to assess whether the applicant carried out the planned data transformations systematically and consistently.
The policy requires the anonymised or redacted reports not to differ in content from the reports for primary use. Not all reports published in 2016 and 2017 said whether this requirement was met. A review of those reports recommends that a report describe any technical change, such as to formatting, pagination or hyperlinks, that the anonymisation process caused.
The techniques the published packages used
The techniques named in the reports of packages published up to 29 March 2018 fall into five categories:
- redaction, covering redaction and masking
- preserving pseudonymisation
- randomisation, covering random offset dates and random values within the study inclusion criteria
- generalisation, covering age generalised to five-, ten- and twenty-year intervals and medical history terms generalised to high level term, high level group or system organ class
- suppression, covering suppression and replacement with alternate text
The use of techniques other than redaction, such as transformation or generalisation of identifiers, was generally limited. Of the 64 anonymised packages, 57, or 89%, used redaction alone for direct identifiers, and 57 used redaction alone for quasi-identifiers. The other seven combined techniques. For direct identifiers associated with trial individuals, six used redaction with preserving pseudonymisation, and one used redaction, preserving pseudonymisation and suppression. For direct identifiers associated with staff information, all seven combined redaction and suppression. For quasi-identifiers associated with trial individuals, six combined redaction, suppression, generalisation and randomisation, and one combined redaction and randomisation.
All seven of those packages applied preserving pseudonymisation to the subject identifier. Screening, accession and manufacturer control numbers were pseudonymised in two packages each, and patient, treatment, sample, custom and pharmacogenomic identifiers, initials, lot, ticket, kit and sample reference numbers, data clarification form identifiers, barcodes and photos of trial individuals in one each. Redaction was used for some of these identifiers in one or two packages where the identifier was embedded in an image, and an experiment number and an Oceans identifier were suppressed in one package each. For staff information, all seven suppressed non-investigator staff names and sponsor and staff contact details, five suppressed the site identifier, and all seven redacted signatures. The sponsor name was always retained, and the names of coordinating or site investigators were retained unless they were directly associated with a subject. Staff information associated with a subject identifier was treated as a quasi-identifier associated with that subject. A site identifier was treated as a direct identifier when it appeared with staff but not with a trial individual, and in one package, where it was associated with a subject identifier, as a quasi-identifier of that subject.
Among quasi-identifiers in the seven packages, age was generalised in six, randomised in five and suppressed in three, and medical history was generalised in six and suppressed in six. Dates were randomised in six, suppressed in five and generalised in three. Race and country were suppressed in four packages each, and the site identifier was replaced with a site identifier chosen at random from the study in four and suppressed in one. Ethnicity, height, weight, body mass index, lean body mass, body surface area, waist, drinking and smoking habits, family and social circumstances, family medical history, psychiatric hospitalisation, names, staff names, organisations, state, city and region were suppressed in one to three packages each, and company or contract research organisation addresses were redacted in one. In three packages quasi-identifiers were redacted where they were embedded in images and could not otherwise be transformed, in three narratives were redacted, and in three listings were redacted as well.
More than one technique could be used for one quasi-identifier, depending on the risk of re-identification: age, for example, could be generalised to an interval or replaced with a random value within the inclusion criteria. Generalisation replaced the original age with a random age selected within a five-, ten- or twenty-year interval, the width depending on the individual subject's risk of re-identification, and replaced the original medical term with the text of its high level term, high level group or system organ class. Randomisation replaced the age with a random value within the age range of the study population, and replaced each date with a new date generated from a random offset for the individual, the offset being applied to all of that individual's dates in the study. One review's text gives the PhUSE offset as the algorithm in six of the seven packages that applied randomisation. Its table gives the PhUSE offset for five of the six packages that randomised dates, the sixth adjusting calendar dates against an offset date. Suppression was used for other quasi-identifiers such as race, ethnicity, weight, height and body mass index, and the alternate text was sometimes longer than the original text.
Rare-disease packages
The risk of re-identification would appear to be higher in rare disease research and small populations. Orphan drug applications deal with rare diseases and studies with small sample sizes.
Of the 52 reports of 2016 and 2017, 12 came from orphan drug applications. Six of these used qualitative risk assessment, four quantitative, and two used the first option to establish effective anonymisation and so performed no risk assessment. Five described an analytical approach while only four performed a quantitative risk assessment, and three used the Lexicon Tool Suite for data transformation, automated redaction and data utility metrics. There is no indication that orphan drug applications are more likely to use the qualitative approach because of small study populations.
Techniques other than redaction were used most often in orphan drug reports. Every anonymised generic and biosimilar package used redaction alone for direct identifiers, and all but one used redaction alone for quasi-identifiers, the exception being a generic package that combined redaction with randomisation, adjusting calendar dates against an offset date. Ten of the 13 anonymised orphan drug packages used redaction alone for direct identifiers. The other three amount to almost half of the seven packages that used redaction together with other techniques. For direct identifiers associated with trial individuals, two of them combined redaction and preserving pseudonymisation and one added suppression, and all three used redaction and suppression for direct identifiers associated with staff. For quasi-identifiers, all three combined redaction, suppression, generalisation and randomisation.
Advice during preparation
Among the lessons one holder drew from preparing its first report are that legal advice is important before the methodology is chosen, to ensure that no data privacy law is breached, and that statistical advice is crucial if a quantitative methodology is chosen.
During the policy's implementation phase the EMA offered advice, by telephone conference, face-to-face meeting or written exchange, to every applicant submitting its first report under the policy. In one applicant's experience the EMA was interested in discussing the problems encountered and the applicant's opinion of the policy, and its feedback improved the quality of the report.
The agency's comments on the anonymisation report
The agency does not formally adopt the anonymisation report, and the anonymisation approach is the holder's responsibility. In its detailed comments on Teva's reports for two dossiers, made during the pilot phase for the larger one and with the redaction conclusions for both, the agency insisted above all that each report be dossier-specific and list only the personal identifiers that actually occur in that dossier. Making that change took time, because it had not been built into the redaction process for personal data and required an additional search of all the documents. Not all of the agency's feedback on the reports was consistent across the two dossiers. The agency also asked for replies to its comments and further changes to the reports after the final redacted packages had been submitted, which the guidance then current did not lead holders to expect. In the holder's view, this bore out the agency's announcement at an industry associations' webinar in January 2018 that it would focus more on the quality and specificity of anonymisation reports in 2018.