Modelling and simulation reports
Pharmacometric modelling and simulation analyses describe and simulate how a drug behaves in patients, and their report presents the models, the evidence of their validity and the simulations run on them, for decisions in development and for regulators.
What the models are for
Pharmacometrics, an emerging field in drug development, combines information from biology, physiology, pathology and pharmacology into mathematical models that can quantify the interaction between drugs and patients. Pharmacometric analyses generate mathematical models that can describe and simulate the pharmacokinetics and pharmacodynamics of drugs. Models of drug concentrations or responses over time show how the amount, frequency and duration of dosing affect drug concentration and how drug concentration relates to pharmacodynamic response. They also help to characterise the pharmacokinetic and pharmacodynamic variability of drugs and the clinically relevant factors contributing to it.
Modelling and simulation of pharmacokinetics and pharmacodynamics are used to inform drug development, support regulatory assessments and trial design, and extrapolate predictions for specific populations. This information can contribute to better patient care and support regulatory decisions.
Model-informed drug development (MIDD) is the strategic use of computational modelling and simulation methods that integrate nonclinical and clinical data, prior information and knowledge, such as drug and disease characteristics, to generate evidence to guide decision making during drug development and regulatory evaluation. Its approaches give a quantitative framework that maximises and connects all the information obtained on a drug during development, enabling that data to be extrapolated to unstudied situations and populations and allowing better characterisation of the drug's risk–benefit profile. MIDD is used, among other aims, to support regulatory assessments, trial design, dose selection and extrapolation to special populations. It aims to expedite drug development, enhance regulatory science and produce benefits for patients.
The extrapolation to new situations that MIDD allows includes:
- support for the safety and efficacy of doses not studied
- prediction of behaviour in special populations, such as the elderly and children, and people with renal or hepatic impairment
- avoiding unnecessary studies, such as drug–drug interaction and bioequivalence studies
- combination selections
Modelling in regulatory guidance
The role of modelling and simulation in drug development and regulatory assessment has grown beyond internal decision-making within pharmaceutical companies into what is now called model-informed drug development. This expanding role has led more pharmaceutical companies to apply these analyses to complement their submission packages, to inform the later phases of drug development, or both.
MIDD is not new. The ICH first put it in context in Guideline E4, and in the 1990s it was largely used experimentally to support development programmes but was not pivotal to decision making. The benefit of modelling and simulation is shown by their integration into regulatory guidelines and by the creation and continuing development of good-practice documents. Population pharmacokinetic, pharmacokinetic/pharmacodynamic and exposure–response analyses are embedded in drug development and are a critical part of many international regulatory guidance documents and frameworks. In some cases the authorities have used modelling and simulation studies to approve a variation of indication even in the absence of clinical data, as in paediatric studies.
The EMA has published papers on MIDD, drafted guidelines that discuss modelling and simulation approaches, created a working party, and hosted workshops that promote the use of MIDD in dose-finding. It has produced guidance on MIDD approaches in paediatric drug development, which overlaps significantly with rare diseases, in drug–drug interaction risk assessment, in renal and hepatic impairment, in obesity and in pharmacogenetics.
The FDA has incorporated MIDD into its regulatory guidance and review processes. Its MIDD Paired Meeting Programme, part of the Prescription Drug User Fee Act (PDUFA) VII commitment, is designed to promote early interactions between drug developers and the FDA on the use of modelling approaches to support a drug's development. One assessment holds that the programme will undoubtedly further facilitate and increase the use of MIDD in rare-disease research.
Regulatory agencies such as the EMA and the FDA have been collaborating with the ICH and focused working parties to develop harmonised guidance that optimises the use of MIDD. The overarching ICH M15 guideline, General Principles for Model-Informed Drug Development, was endorsed by the ICH assembly for public consultation as a draft dated 6 November 2024. It aims to facilitate greater and wider adoption of MIDD principles in drug development and regulatory decision making across the major ICH regions of Europe, Japan and the US, and among the other regulatory and industry members and ICH observers such as the WHO.
Requirements on the report
In pharmacometrics the documentation and reporting are regulated as well as the analysis, and the documents produced need to conform to specific requirements. Original data files, data transformations and their code, and the computation and coding of the final model and simulation files all need to be made available. Data and results need to be shown in specific types of plot, and the validity of the models must be demonstrated with suitable model diagnostics. A white paper from 2016 gives an extensive overview of the guidelines and good-practice documents and of the published recommendations for model building and its documentation.
Time is very often key for these analyses. When they have to be performed immediately after clinical data become available, to inform an internal decision or to support a regulatory submission, results are expected within short timelines. Analyses for regulatory submission combine short timelines with large input datasets and extensive output files, all of which need to be tracked, organised and interpreted in regulated reports. Reporting them can be technically challenging, given the amount of input and output data to be summarised and accurately described, so reproducibility, automation, traceability and standardisation are considered key aspects of the process. Like many other areas of regulatory writing, the reporting benefits from some automation and standardisation, and from a clearly organised, structured and reproducible system.
From data to submission
Pharmaceutical companies either perform pharmacometric analyses in-house, when their competences and resources allow, or request them from specialised contract research organisations. The typical main steps of the analysis and its reporting, with examples of the actions performed in each, are these:
| Main step | Examples of actions performed during the step |
|---|---|
| 1. Data transformation | Create a variable that groups subjects by type of underlying disease |
| 2. Data exploration | Observe trends in the data, such as subjects with a specific disease type eliminating the drug faster |
| 3. Model building | Develop a model that describes the pharmacokinetic profile of a drug while taking into account possible sources of variability, such as type of underlying disease, age and genotype |
| 4. Description of the model results | Results are generated by external software, such as NONMEM; they are then analysed to demonstrate the validity of the models, for example by generating plots such as visual predictive checks, and to draw conclusions on the endpoint analysed |
| 5. Simulations | If the results are further used to perform simulations, these allow predictions of how a drug is expected to behave, for example in specific patient populations |
| 6. Creation of submission-ready reports | Reports are generated, peer reviewed and quality controlled; when finalised, the report and all supporting documentation are included in an e-submission package that is ready for regulatory submission |
When a contract research organisation performs the analysis for a pharmaceutical company, a typical project starts with discussions with the client about the objectives, the project planning and its definition, to reach agreement on the analysis plan. When data from clinical studies become available, the client delivers large data files, possibly in different formats, which are explored and transformed into data files that modelling software such as NONMEM can read as input. Pharmacometricians then analyse the data, develop models that appropriately describe them, and possibly perform simulations in accordance with the purpose of the analysis. This hands-on work generates large amounts of output files and data in different formats that should be summarised and interpreted, so during the analysis modellers need to gradually put the information together in a clear, structured and understandable way. By the end of the analysis, it needs to be organised in a report that is consistent with the standards of the contract research organisation and its clients and conforms with the content and quality requirements of the regulatory authorities.
Generating the report from the analysis code
One contract research organisation generates its reports from the analysis with a combination of the statistical computing and modelling program R, with its integrated development environment RStudio, the LaTeX typesetting system, and knitr, an R package that integrates R code into LaTeX. The combination is one of many ways to achieve the main goals of consistency, traceability and standardisation of reporting. It makes the whole process after data collection traceable and reproducible, in line with the principles of reproducible research. A reproducible system is one that, starting from identical input data analysed with the same methods, should give the same output and an essentially identical report, so the final report can be more in line with internal organisational standards and less dependent on the individual.
In the traditional workflow, analysis and reporting are consecutive steps, and review feedback has to be implemented manually for every step. Integrating analysis and report generation creates a seamless chain between raw data and final report, in which review feedback is incorporated and data errors are corrected in one place and then propagated automatically throughout the report. In the input, LaTeX syntax and R code stand together, and in the output PDF the text comes from the LaTeX syntax and a figure from the R code. When RStudio receives the command to compile a PDF, it automatically:
- executes the R code and replaces it with the appropriate LaTeX code
- typesets the LaTeX document into a PDF
- updates the bibliography numbering and reference list from the information in a BibTeX file
- updates the glossary and includes all abbreviations according to the company standards
The reporting system is the point at which scientific analysis, automation and scripting, and medical writing meet. For the user, each report section consists of three blocks:
- instructions on which specific information to include, how to include it and which output files to append, so that the report delivers structured content
- section-specific verified scripts and functions that generate standard figures and tables in accordance with regulatory requirements, using R packages validated to comply with good clinical practice
- standard text that helps describe and clarify the methods and processes typically used, with optional standard text for the most common alternative analyses
The full product is a PDF of a few hundred pages, ready for regulatory submission. Where a leaner document better fits the purpose, the length and subsections of the report can be tailored to the client's request by selecting the document template and the type of analysis before report writing begins, and slide decks can be produced with the same system.
While the system needs to be solid and standardised to comply with regulatory requirements, it also needs to be fit for purpose and flexible enough to adapt to the type of analysis. Examples are a pharmacokinetic–pharmacodynamic analysis of an orally delivered drug and its active metabolites or of an intravenously infused drug, and one of a time-to-event endpoint or of an endpoint measured as a continuous variable. Setting up the system, defining its technical details and creating user-friendly instructions for each step requires a multidisciplinary team of pharmacometricians, system developers, data programmers, medical writers, quality control reviewers and others. Despite its technical complexity, the system is rather straightforward to use once set up. It makes use of R's literate programming capabilities, under which it delivers a user-friendly PDF and is also more robust and easily maintained. When users identify pitfalls, or new methods or software updates are released, the system can be updated and refined, and new versions released in which standard code is adapted and dependencies across programmes are revised.
Advantages
The advantages of the system concern the technical side of the process and the characteristics of the final document. Although the input comes in several file formats, the product is a PDF, which is often the format clients prefer for final reports. A PDF does not allow accidental modifications and can easily be signed with official e-signing software. The software is open source and so available to everyone at no cost, is not specific to pharmacometrics and so can profit from developments in other fields, and can handle large and complex technical documents.
The system allows reproducible reporting and traceability of data sources, data transformation and analysis. Validated R scripts are more efficient and much less error-prone than typing all the content manually and creating plots and figures from scratch for each new project. With coded content, an error can be corrected by changing a value, code, file name or directory only once in the master document, and code dependencies propagate the correction through the rest of the report. These factors are known to reduce the overall time and money spent on performing, writing and quality-control reviewing such analyses.
The system also delivers structured content, a document in which information is placed in the appropriate section, which some authors have called labour-intensive and the most time-consuming, tedious task for a medical writer. Besides standard text and instructions, scripts that retrieve information from different parts of the document, or from separate documents, and combine it in the appropriate sections contribute to it, with only minor manual adaptations needed. Guiding text and standard sentences for alternative scenarios make formatting, tone and language style more consistent across scientists, which also supports professionals less focused on the linguistic aspects of reporting. So a medical writer needs to spend less time rephrasing entire paragraphs to adjust language style and formatting.
Drafting before the final data
Reports can be prepared even before clinical trial data are accessible and before the final models are generated. In a preparation phase the main analyses can be performed, for example with dummy data or previously published data from a similar study, and the report can be shaped and partly quality-control reviewed in the relevant sections. When the final study data become available, most of the effort can go on the outcome-related aspects that need to be interpreted and reported. The preparation phase shifts the time spent on report writing from the final, time-critical stages to an earlier and more convenient phase of the analysis.
In one case, a pharmaceutical company had pharmacometric analyses performed on data from a phase III study of a drug used for cardiovascular diseases. Much of the work was done in the preparation phase: planning the analysis in detail, creating data files of dummy data, generating a dummy report with simulations to prepare for several alternative study outcomes, and performing scientific review and quality control of the analysis and the report. Once the study was completed and the final data were available, scientists could spend their time on the actual data analysis rather than on extensive writing and editing of the report. The modelling and simulation report went from final data access to regulatory submission in 7 weeks.
Drawbacks
The main drawbacks of the system relate to the technical complexity of its tools. Less IT-skilled professionals may find the interface rather unfamiliar and somewhat archaic. Most users may be largely familiar with Microsoft Word, but Word does not allow the programmatic integration of text, plots, tables and abbreviations generated with standard code, which LaTeX allows in combination with R and knitr. R Markdown is more user-friendly than LaTeX but may not produce PDF documents meeting all the requirements of regulatory agencies, so LaTeX remained the preferred software. Users need to understand most of the R code that generates tables and figures to adapt it when a function has to change, as when a standard plot shows subjects' median drug concentrations over time and the client asks for the mean values instead. The document generated for revision and finalisation is a PDF rather than a Word file, and users more familiar with Word's review functions may need some training to add comments and revisions to a PDF. These technical aspects, together with the need to learn and adapt to the company-specific standards of analysis, make the learning curve steep for those using the system.
R, RStudio, LaTeX and knitr also need to be correctly integrated into the IT environments already in use at the company, and any process adaptation or program update needs to be compatible with the rest of the system. The main drawback of the final document is probably that it is more template-oriented than project-oriented, so particular client requests or project-specific needs sometimes require additional effort to implement. The whole system needs to be accurately installed so as to protect business confidentiality, information security and access to confidential regulatory documents. The organisation that developed the system holds that, despite these challenges, it has increased the efficiency, accuracy and reliability of its work and contributes to the reproducibility, automation, traceability and standardisation of its deliverables.
Modelling in rare-disease dossiers
Development of new drugs for rare diseases is one of the pivotal areas in which quantitative modelling is used extensively. MIDD can be applied to all therapeutic areas, but rare diseases have a greater need for it because fewer patients are available for study. With an inherently small pool of people with a specific rare disease, few patients enrol in clinical trials. Often no more than one pivotal phase 3 study can be run, and that one is usually of small sample size. Early phase studies frequently give insufficient data to inform dose selection for later phase studies, and dose optimisation studies can be unfeasible. Natural history data and real-world data are often limited in rare diseases. The challenge of generating adequate evidence where information is limited, as in rare diseases, has gained visibility.
Maximising the use of all available data about a new product is therefore paramount in rare-disease drug development, and it brings opportunities for MIDD. MIDD approaches allow the integration of all available data, including nonclinical studies, controlled clinical trial data, observational data and aggregated literature data, into a totality of evidence that enables a more robust characterisation of the drug's risk–benefit profile. Further data sources, such as patients' electronic health records, genetic data and patient registry information, can be leveraged in MIDD to further the understanding of the disease and the investigational treatment. The models allow prediction of responses and inform efficient clinical trial design in diseases with scarce patients.
Particularly since 2021, submissions for rare diseases have come to rely heavily on the modelling and simulation provided as part of MIDD. Modelling used to be a supplemental part of the dossier, and regulators now expect its inclusion.
Writing about modelling
Modelling and simulation are used in all phases of drug development in regions across the world, and they have historically been used most frequently to support the clinical pharmacology files and labelling for new drug applications. Since regulators expect modelling to be included in the files so that all available data can be scrutinised, marketing applications in the EU and US are rarely submitted without modelling being used to describe the pharmacokinetics of a new medicine, especially where clinical data are scarce. MIDD affects all types of regulatory document a medical writer may encounter, from protocols to clinical study reports and summary documents, and ultimately product labels.
Medical writers are most likely to come across population pharmacokinetic and pharmacodynamic models, the most prominent class of pharmacometric models used in clinical drug development. Familiarity with the documents and outputs of this modelling, such as the plans and reports of population pharmacokinetic and exposure–response or exposure–safety analyses, is integral to working on rare-disease submissions. Medical writers are highly likely to encounter additional models and their documents in the near future, so keeping abreast of the topic is important. The draft ICH M15 guideline refers to modelling analysis plans and regulatory reports, and to MIDD assessment tables, for communication within and between drug developers and regulatory authorities.
MIDD is a highly collaborative process, involving statisticians, pharmacometricians and multidisciplinary teams. Regulatory medical writers communicate the results of often complex modelling and simulation exercises to decision-makers and upper management in the pharmaceutical industry and to multidisciplinary review teams within regulatory agencies. Regulatory medical writing translates complex medical and scientific information into comprehensive yet concise and consistent text, a skill that is particularly vital in MIDD, with its complex nomenclature and multifaceted data outputs. Often only medical writers with a specific background or interest in pharmacokinetics are considered for writing the parts of a submission that explain MIDD and present modelling and simulation data.