Producing the report's patient narratives
A clinical study report's patient narratives are produced from more than one database, mostly once the final data are available, and sometimes in large numbers. Their production turns on which data they rest on, how physicians and writers divide drafting and review, when the work falls, and how far programming, automation and AI can take over the work.
The data the narratives rest on
Clinical trial data are captured in two separate databases. The pharmacovigilance database captures the safety data from which the pharmacovigilance or drug safety group produces pharmacovigilance safety narratives throughout the study, and the clinical database captures all trial-related data, safety data included. Once all the data are clean, the two databases should ideally be reconciled.
The report's narratives are event-based and rest on clinical data. Pharmacovigilance safety narratives are subject-based, rest on pharmacovigilance data and are written on Council for International Organizations of Medical Sciences (CIOMS) forms, which are useful in part for preparing the report's narratives. At reporting, data from the clinical database take precedence over data from the pharmacovigilance database, so inconsistencies between the pharmacovigilance narratives and the report's narratives must be reconciled.
Some companies routinely reconcile the two databases, generate final CIOMS forms from the reconciled database and place those forms in the report in lieu of separate report narratives. The narratives then align with the clinical data, but they remain subject-based rather than event-based, and the approach may confound identifying the event or events that need a narrative.
Drafting and reviewing serious adverse event narratives
One process that assures high-quality serious adverse event narratives starts from a template based on ICH E3 and sets out the procedures from initial drafting to final incorporation into the report. The study's assigned medical monitor evaluates the serious adverse events as they occur and drafts the narratives from the clinical data and the pharmacovigilance data. A team of two to three clinical development physicians and a pharmacovigilance representative then reviews the drafts. The review may raise further queries, which the responsible medical monitor resolves with the clinical study team or directly with the investigative site, and all responses to queries are shared to update both databases. A medical writer then processes the drafts and verifies their data further against the final study data, and after a quality control review the narratives are incorporated into the report.
The process combines the medical monitors' medical expertise with the medical writers' technical writing skills. The events are investigated proactively, and the site is queried as close as possible to the time of the event. Because all medical monitors take part in the medical review, the narratives are consistent across therapeutic areas, and the process also helps train new medical monitors.
When the narrative work falls
Most of the work of generating a report's narratives has to happen once the trial's final data are available, after database lock and once the report's tables, figures and listings have been generated, and the narratives must then be final and ready for the final report. Doing that work efficiently and within the required timelines takes strategies that reduce time and effort and increase efficiency.
The workload of writing narratives
Generating a report's narratives can be a complex, time-consuming and costly task. Narratives are onerous to write, especially where there are many variables or many participants with adverse events, as in an oncology study. One report can require 1,000 narratives or more, depending on the phase of the trial, and the work of generating a narrative for each participant can call for a large team of dedicated medical writers.
In treatments for rare or hard-to-treat conditions patient narratives can run to tens of pages, so a writer can spend days drafting a few, and studies with more participants take more effort still. Before the writing starts, statistical programmers and others prepare the files the writer works from, a job that can take weeks or months and, because the data sets are so large, is prone to human error.
Before any form of automation, narratives were created by hand: the writer started from a template and filled it with data from the various listings provided for the report. Retrieving the data from each listing for each narrative took a long time and increased the chance of human error, so every item of data in the narrative needed a full, thorough quality control check. A narrative can become much more complex where the trial participant has a complex disease status, because describing the disease or clinical status, the concomitant medications and the course of events takes a large amount of information. When a report required hundreds of narratives, the manual process was a huge undertaking, and so created the need for a more streamlined approach.
Writing patient narratives from clinical study data is often a manual, tedious and time-consuming task for medical writers, safety specialists or both. A narrative typically reports demographics, baseline characteristics and medical history, adverse events and serious adverse events, and laboratory values. These data come from statistical outputs such as tables and listings and are usually copied into the narrative by hand. The study team also has to review the outputs by hand to find which patients meet the study-specific criteria for a narrative. Together these add cost, resources and time, both to develop the narratives and to verify their data in a quality check.
Programming narratives
Best practices for creating a report's narratives have evolved over time and in September 2023 were still evolving, with processes becoming increasingly efficient. At that time the process-driven automation approach was widely adopted. It fills a narrative template whose placeholders or fields receive data from the clinical database programmatically. Medical writers are not trained programmers, so they do not write the code themselves but use software or the help of trained programmers. In September 2023 the automation tools for writing narratives did not use machine learning: they used preprogrammed natural-language text into which specific data items were imported from the data set generated by the electronic data capture systems used in clinical studies.
In a large organisation with a department of programmers, the writer assigned to the narratives can work with a programmer to tailor the narrative template to the trial. The writer sets the rules in the template for which action follows, an automated trigger or a manual intervention, and the programmer can run narratives whose text-based sentences are populated with the data points required from the trial data captured in the case report form fields. The programmer can generate as many narratives as the report needs, with consistent and accurate outputs. Alternatively, the writer can load the clinical data into purchased software and set the criteria for the narratives: which narratives are run and which data they include, and so how they look. At the click of a button, the software generates the required narratives.
Because the data are inserted programmatically, the writer does not need to quality-control check each item of data, but the writer still has to intervene by hand, whether the narratives come from a programmer or from software. The clinical database is the primary data source, but supplementary data from the safety database, including CIOMS or MedWatch forms, still have to be inserted by hand. These data add detail to the story of the events, and the text added by hand merits a quality control check against the programmed output. Automation still clearly saves writers time and effort in generating large numbers of narratives.
A programmed narrative still demands work. Editing a machine-generated narrative into coherent text with a clear story is substantial work, tailoring the program to a product, study and population takes significant programming effort, and medically complex cases often need information from outside the electronic data capture system, such as CIOMS forms.
Automation has filled a gap in writing routine text such as the report's safety narratives. FDA reviewers have not favoured safety narratives written entirely by automation, so such narratives are less common than they might be. Hybrid narratives, with the data in brief tables and the discussion and assessment written by a medical writer, can be efficient, accurate and medically useful.
Where large numbers of narratives are to be written, programming subject profiles is a cost- and time-efficient approach worth considering. A subject profile gathers the narrative-relevant line-listing data for one subject into a single file, similar to the US archival listings. Selected tabulations, of laboratory data for example, break up the prose and aid comprehension of a narrative, and partly tabular narratives also improve project efficiency.
One interdepartmental process programs the narratives' headers. The headers carry the same information for each subject, such as demographics, medical history and concomitant medication, which is often painstaking to pull out of the listings by hand. Used on narrative projects of 25 to 1,200 narratives, the process allowed large numbers of narratives to be created more efficiently and with fewer data errors, and larger projects to be taken on and completed on time.
A narrative-generation utility is software that generates patient narratives automatically in Microsoft Word. Using one may bring these benefits:
- Study-specific criteria, such as adverse events of special interest, are predefined and decide which patients need a narrative, so the author does not review adverse event listings by hand and no qualifying patient is missed.
- Predefined data points, such as baseline information, are populated from the statistical outputs into each patient's narrative template, in an order and layout configured for each study, so the author can focus on the descriptive text rather than search the outputs and copy the data.
- Quality-control reviewers do not need to verify every data point, and review takes less time because sections are populated from validated data, which reduces human error.
- All the narratives for a study can be compiled automatically into one submission-ready document, based on the list of patient narratives.
- Narratives for a study can be regenerated consistently.
- After database lock, the utility can be re-run and the new narratives compared with the originals, with differences highlighted for the author to accept or reject.
Implementing such a utility can bring significant time savings and better quality. Relieving authors and quality-control reviewers of manual work lets them focus on their areas of expertise rather than mundane tasks, and may even make a happier workplace.
Writing narratives with AI
In September 2023 AI was a promising tool for automating narrative writing beyond the programmed approach. AI tools fall mainly into machine learning, which analyses structured data and selects the information wanted, and natural language processing (ChatGPT is one form), which extracts and analyses information from unstructured data such as clinical notes to enrich the structured data. An AI tool can automatically interpret the type of input data, such as case report forms, clinical database listings, and CIOMS and MedWatch forms, and generate a full narrative itself. AI can save writers more than 30% of their quality-control time and up to 80% of their time overall. Two medical writers hold that AI that integrates natural-language sources with coded, cleaned and meticulously queried databases will reduce much of the writers' routine work and make narratives much more useful to reviewers and other stakeholders looking for potential problems with a product.
Preparing initial drafts of patient narratives with AI first requires establishing relationships between the relevant data points within complete databases, and only then can the drafting be automated with natural language processing and natural language generation. Combined with well-structured data modelling, natural language processing and natural language generation not only process the data but also highlight the relationships and key components within it, which gives the narratives higher accuracy. This approach may save writers hundreds or even thousands of hours and cut submission cycles significantly. The use of AI, natural language processing and natural language generation in automating regulatory submissions such as clinical study reports is meant to augment writers by helping them create initial drafts and revise them in much less time; without it, writers spend a significant amount of time preparing the drafts and updating them at the sponsor's request.
The benefits of AI in producing narratives are these:
- It makes writers more efficient.
- It raises the overall quality of the documents.
- Reusing the tool to generate narratives after the first project can reduce overall costs.
- It reduces the risk of errors.
- It reduces the number of reviews required.
- It keeps the narratives consistent with each other.
- The tool can be tailored closely to a project's requirements.
- It lets the writer focus on interpreting the data and on messaging.
Its potential challenges are these:
- AI detects and analyses patterns, so it needs training on large volumes of content to generate human-like text.
- It has to comply with laws and regulations, particularly privacy law such as the GDPR.
- It has to comply with AI-specific law, such as the EU AI Act, which in September 2023 was in the process of being enacted.
- Ethical issues such as bias in algorithms and protection of patient privacy remain largely unaddressed.
- Text generated by AI tools rests on patterns rather than facts, which often results in factual errors.
- The writer using an AI tool needs a new skill set.
- Selecting only the relevant data may be hard, because the tool may include all the data rather than just the relevant data.
- The narrative cannot be fully automated: the writer's input is still required to ensure the highest-quality narratives.
- AI brings no efficiency to the medical review of the narratives.
Safety narratives written entirely by AI need a large data set to train the algorithms to produce adequate text.
Producing narratives with AI still starts from a template that the writer adapts to the trial. That early investment of time saves time when the AI tool generates the narratives, and for later trials that follow similar client or product specifications, generation is even more efficient because the template needs less work. In September 2023 the use of AI to generate narratives was still emerging, and the programmed approach was still well suited to standardised, repetitive documents such as narratives.