Recording and managing data

The FAIR data principles

7 min read

FAIR data are findable, accessible, interoperable and reusable by people and machines. The principles govern data and metadata without requiring that the data be open or free.

Purpose and development

The fifteen FAIR principles were published in March 2016, following a Lorentz Centre workshop a little over two years earlier. They address the technical handling of digital data and services, particularly their use by computers acting autonomously. They apply across domains, including clinical trial and healthcare data, and cover raw data and the metadata that enable discovery, linkage and reuse across systems.

Combining data from many sources creates value, but finding and manually processing those data can make reuse impractical. FAIR addresses the argument that data production exceeds the capacity for manual integration, limiting returns from investment in big data, analytics and artificial intelligence. A 2018 European Union cost-benefit analysis estimated that the absence of FAIR research data cost the European economy at least €10.2 billion annually.

By 2020, the G20, G7 and World Economic Forum had endorsed FAIR, and the European Commission had announced the European Open Science Cloud as infrastructure based on the principles. Comparable infrastructure initiatives had emerged in the United States, China and Africa. The principles' publication had also helped advance data sharing, particularly in academia. FAIR originated largely in the life sciences and has often been driven by biomedical applications, giving medical writers a role in correcting misconceptions.

The fifteen principles

The principles divide into findability, accessibility, interoperability and reusability.

PrincipleRequirement
F1Data and metadata have a globally unique and persistent identifier.
F2Data have rich metadata, as defined by R1.
F3Metadata clearly and explicitly include the identifier of the data they describe.
F4Data and metadata are registered or indexed in a searchable resource.
A1Data and metadata are retrievable by their identifier using a standardised communications protocol.
A1.1The protocol is open, free and universally implementable.
A1.2The protocol allows authentication and authorisation where necessary.
A2Metadata remain accessible when the data are no longer available.
I1Data and metadata use a formal, accessible, shared and broadly applicable language for knowledge representation.
I2Data and metadata use vocabularies that follow FAIR principles.
I3Data and metadata include qualified references to other data and metadata.
R1Data and metadata are richly described with multiple accurate and relevant attributes.
R1.1Data and metadata are released with a clear and accessible data usage licence.
R1.2Data and metadata have detailed provenance.
R1.3Data and metadata meet domain-relevant community standards.

Common misconceptions

Four distinctions prevent common misunderstandings of FAIR.

  1. The principles are directed at machine agents, although their ultimate purpose is to help people solve complex problems and make better-informed decisions.
  2. FAIR is a set of principles that can make formats, protocols and standards machine-actionable, rather than a format, protocol or standard itself; agreement on the principles does not settle how to implement them.
  3. FAIR does not mean open or free: conditions of access must be clear enough for a machine to understand, but access may require permissions, a restrictive licence or a fee.
  4. FAIR makes no explicit claim about data quality, which depends on the intended use; it requires provenance that helps users assess quality.

The open, free protocol required by A1.1 can therefore support access to protected data. Patient records can be FAIR while remaining closed under privacy rules, and data available only to qualified researchers after panel review can also be FAIR. Metadata can make the privacy restrictions explicit. Repositories can support the aim of making data as open as possible and as closed as necessary.

Rich, machine-readable provenance metadata are often missing from conventional datasets. Provenance can record when and where data were produced, by whom, how acquisition was funded, and which methods and instruments were used to collect and analyse them. Such metadata can improve assessments of trustworthiness and fitness for use, without guaranteeing quality for every purpose. Extensions such as FAIR-ER have been proposed to add ethics and responsibility explicitly.

Implementation and data visiting

GO FAIR was established through joint public funding from ministries in the Netherlands, Germany and France to support convergence on FAIR standards and technologies from the bottom up. By 2020, implementation involved research projects, initiatives and commercial services.

Fine-grained access control can permit certified algorithms to use selected elements, such as blood pressure measurements, while withholding identifying information. In the proposed FAIR Data Trains approach to distributed learning, algorithms visit data held at FAIR Data Stations and receive only the access permitted for that use. This data visiting approach could let owners, potentially patients themselves, authorise analysis without relinquishing control of their data. In 2020, the Virus Outbreak Data Network, involving public and private regional authorities, was attempting to establish an international network of FAIR Data Stations containing COVID-19 electronic case report forms for automated data visiting.

FAIR pharmaceutical publication data

In 2022, pharmaceutical publication data fell short of FAIR principles in several respects, and medical writers could help close those gaps.

Findability

Aggregated and summary data were relatively easy to discover through ClinicalTrials.gov and publications, but their links to the underlying raw data were not always obvious. Whether company databases holding data such as non-interventional study data met F1–F4 was often unclear. Shared patient-level trial data repositories included Vivli, ClinicalStudyDataRequest.com and the Yale University Open Data Access (YODA) Project. Vivli assigned each dataset a digital object identifier (DOI), meeting F1, and offered searches by population, intervention, comparison and outcomes, although its catalogue did not yet support highly precise searching and browsing.

Journals with stricter sharing policies required persistent identifiers and recommended appropriate descriptions, metadata and deposition in searchable repositories, supporting F1–F4. Writers can prepare metadata and documentation for secondary users and may advise on repositories for less common data types.

Accessibility

Privacy and safety concerns, data management and curation costs, and possible publication delays complicated access to patient-level data. YODA published common reasons for refusing requests, including limits in informed consent. Vivli retained data for up to ten years and maintained a dataset's DOI after the data became inaccessible, supporting A2, whereas continued metadata availability in other repositories was less clear. Journals mandating sharing often recommended external repositories, supporting A1.1 and A1.2, but typically did not require A2; some Taylor & Francis journals required fully FAIR data. Writers can help prepare protected data for access and explain barriers to authors.

Interoperability

Interoperable data can be integrated with other data, applications or workflows for analysis, storage and processing. In 2022, companies were increasing efforts to structure and annotate data, and interoperability was a stated aim of Vivli, yet reuse and reanalysis across systems remained difficult even when data were accessible. The COVID-19 pandemic prompted calls for greater interoperability across the industry.

Writers can help by using standardised formats, controlled vocabularies, machine-readable text rather than PDFs, and open formats such as CSV rather than proprietary spreadsheet formats. Clearly annotated table headings, scales and other elements, and repositories that support interoperability, can also make reuse across systems easier.

Reusability

Increasing the community's ability to use data and metadata is FAIR's primary objective. In 2022, compliance with the clear licensing requirement of R1.1 across pharmaceutical publication data was uncertain; the Pistoia Alliance recommended human-readable and machine-readable metadata pointers to the data owner or licence. Journal requests for protocols and statistical analysis plans, and publishers' requirements for data citations, supported provenance under R1.2. Datasets accompanying pharmaceutical publications and datasets submitted to established repositories typically met domain standards under R1.3. Unusual or complex datasets could be harder to standardise because curation takes time and money; recognising and rewarding curation could improve adherence.

Further improvements

Legal, privacy and safety restrictions can prevent pharmaceutical publication data from being open while leaving scope to make them FAIRer. Proposed improvements in 2022 included standardising data structures, metadata and systems; unifying policies across regulators, publishers, companies and other data producers; giving clearer metadata guidance that reduces management burdens; and educating authors about FAIR data. Writers can promote these practices through their work with academic authors, pharmaceutical companies and publishers.

Sources behind this topic

2 sources · 15 passages

Follow the evidence

Supporting passages

The source text behind this topic. Read it in context.