Working with biostatisticians
A medical writer and a biostatistician bring words and numbers to the interdependent documents of a clinical trial, and neither can finish their part without the other. They work together from before the first data are collected, through the study concept, the protocol, the analysis plan, the statistical outputs and the clinical study report, until the report is filed, and outside a trial a writer may read a publication with a statistician.
Why the two need each other
The biostatistician and the medical writer play key roles in a trial without visiting a site or seeing a patient, and in spite of their back-office positions their roles are crucial to study design, conduct and data analysis, because they deal with a wide range of interrelated trial documents. Generally the writer's core competencies lie in producing words and text and the statistician's in numbers and the analysis of data, and though the two skill sets look divergent at first glance, they have a powerful synergy that can have a major impact on how a trial is executed.
One medical writer and journal editor holds that to do their jobs well medical writers must be able to communicate and collaborate effectively with biostatisticians, whom he still thinks an alien species that speaks a different language. In one columnist's view, communication between authors and statisticians can often be very challenging, because the authors speak their clinical language and the statisticians statistical hocus pocus. The medical writer comes in, or should come in, to translate, so that the statistician understands the clinical question and the author comprehends the statistical answer, which is not an easy task. A writing consultant may be required to communicate complex statistical output created by statisticians, who communicate in mathematical notation and statistical terms, to clients whose eyes roll back when they hear statistical phrases. One professor who reviews many manuscripts holds that for ideal cooperation both the statistician and the medical writer need to understand the clinical and biological context and basic statistical principles in order to interpret the results of a statistical analysis properly; mere number crunching is not enough, and continued cooperation and mutual exchange is the key.
In regulatory medical writing, the medical writer and the statistician must work as a team. Biostatisticians and medical writers are among the key people who develop a trial's important documents, and working together as they should can streamline the preparation of those documents and ensure their quality. A successful clinical study provides accurate, reliable and valid data that let regulatory authorities make accurate decisions about the safety and efficacy of the medicinal product, and achieving it requires the multidisciplinary team to collaborate effectively from development of the study concept to data analysis and reporting.
In Russia, where the clinical trial industry and the local regulatory requirements were developing rapidly in 2016, medical writers in contract research organisations had to take on non-traditional roles, providing not only writing but scientific advice on drug development, regulatory affairs, study design and methodology. Those roles required a good understanding of basic biostatistics and of the statistical methodologies applied to clinical trials, and therefore close collaboration with biostatisticians, especially within contract research organisations. Because of new regulations and ongoing change, writers and statisticians in Russian contract research organisations both played non-traditional roles, acting in addition to their usual functions as a source of expert knowledge on the scientific and regulatory aspects of study design and methodology. Close collaboration between them improves the efficiency and quality of the drug development process, and an environment of discussion, support and shared learning can foster it.
Help with statistics should be sought without hesitation, because it is easy to get lost in statistical analysis: one study found wrong or suboptimal statistical tests in 5 of 31 papers (16.1%) in the New England Journal of Medicine, either because a test was incompatible with the data examined, because parametric methods were used inappropriately, or because a test was inappropriate for the hypothesis under investigation.
Involving the statistician before data are collected
In one medical writer's experience, the statistician is frequently involved too late in the research process: approached with an often meticulously collected set of raw data and curtly asked to run some stats on it, the statistician often replies that the study is underpowered by an order of magnitude to show the expected effect. Disappointment is followed by a plan B in which the authors ask the statistician for some mathematically dubious sorcery to shrink the P value, and the medical writer ends up in an equally uncomfortable position, having to word far-fetched conclusions from exotic maths they barely understand. One objectionable trick is generating hypotheses after the fact from the sample available. The relationship between the author, the medical writer and the statistician, not to mention the sponsors, is often underpinned by a power struggle in which each party strives to vindicate its own agenda, and the writer thinks that non-statisticians' resentment of statistics plays an important part in it.
The writer's advice is to always design the hypotheses to be tested in advance and to invite the statistician for coffee well before the first batch of data is collected, to explain what sample size gives sufficient power and which statistical methods suit the purpose. In the writer's view, hypotheses devised in advance spare the statistician dredging the data and the writer describing it, and with a few exceptions they may keep the work within undergraduate-level statistics, with t-tests, linear regression and correlation coefficients dominating and perhaps Mann and Whitney visiting every now and then.
Setting up the collaboration
At the start of the study the two should get to know each other's names, exchange contact details, time zones and work schedules, and discuss timelines; in a digital global office environment, a strong professional affinity and an open line of communication are especially important. Working conditions greatly influence team effectiveness: having most of the medical writing and biostatistics staff in one office eases communication and lets issues be resolved quickly enough to keep project timelines, and regular face-to-face meetings from study start-up, together with shared training on standard operating procedures, build understanding of project plans, milestones and specific project requirements.
The standard operating procedures of the two functions should be aligned and should not contradict one another, and it is best for the two teams to consult each other when developing and revising their procedures and processes; failing that, they should at least share their relevant procedures and process guidelines.
The two should inform each other of what they expect in templates and style guides, the number of review cycles, the expected review time and the level of review, for example text and content only or formatting and grammar as well, and the whole document or only certain sections. Their deliverables are interdependent, so each team has to be aware of the other's timelines and should not agree to a deadline without consulting the other.
The writer should not hesitate to ask statistical questions, even ones that seem basic or have been discussed before, and the statistician in turn should consult the writer on textual, content and formatting issues and, where necessary, for guidance on regulatory requirements. The statistician should also take advantage of the writer's awareness of the target audience to customise a technical document to the level of its intended reader. When team members work closely, professional skills and knowledge are easier to attain, and inside a Russian contract research organisation a collaborative learning environment can be maintained by sharing and discussing useful publications, conference materials and new regulatory information.
The study concept
Developing the study concept is among the most challenging parts of a study and the most important for its regulatory, scientific and financial success. In Russian contract research organisations the writer and the statistician are responsible for ensuring that the study complies with local and international regulatory requirements, applicable scientific guidelines and the trends in the specific therapeutic area, that it is feasible and time- and cost-effective, and that it produces internally and externally valid results. The two must work together to research and select the study population, the primary and secondary endpoints, and the types and time points of study assessments in the context of the requirements for that phase of development. The statistical aspects of the study concept often become particularly challenging when developing a local registrational phase III study for a product that has been registered in other countries for a long time and for which clinical trial data may be limited or absent; such cases usually require extensive literature searches and a great deal of creativity and thought to build arguments supporting the study concept and design.
The golden thread through the documents
The documents the two produce, the clinical study protocol, the statistical analysis plan and its mock outputs, the statistical outputs themselves and the clinical study report, are all interrelated and revolve around common themes: the study design, the study objectives and the corresponding study endpoints, interconnected by what is called the golden thread. Working together, the statistician and the writer need to ensure that objectives and endpoints are well defined, congruent, and consistent across every document as the study proceeds.
Moving content from one document to the next too often introduces errors and loses information inadvertently, for example during copy and paste, and the same can happen while a document is revised. To avoid this, any change to a document and the rationale behind it should be discussed within the team and clearly documented, and both the statistician and the writer should be involved in the review of each document.
The protocol is the main starting point of a trial. In drafting it the two should work together so that objectives and endpoints align: the statistician should complete the statistical sections, including the sample size calculations, and the writer should review them, and any ambiguities should be clarified and any changes that have to be implemented as the study proceeds documented. In Russian contract research organisations the writer is responsible for developing the protocol once the study outline is finalised, and the statistician usually writes its statistical part, which the writer reviews so that terminology, text style and formatting are consistent and follow the appropriate templates and style guidelines. The writer there must therefore understand the main statistical aspects of the study well enough to comment on the statistical methods in the protocol. That includes awareness of the basic approaches to sample size calculation and the regulatory requirements for it, and of the much greater attention the Russian regulator had been giving by 2016 to the details of study design and to the statistical aspects, including the assumptions used for sample size calculation.
The statistical analysis plan is developed early in the trial and is the statistician's responsibility, and the writer, as one of its downstream end users when the report is developed, should be able to review and give feedback on the plan and on the shell, or mock, statistical outputs before they are finalised, to ensure consistency between the protocol and the plan. Medical writing review of the plan before it is finalised is highly recommended, to ensure that the structure of the statistical outputs agrees with the guidance documents, such as the structural guidance of ICH E3, and that every summary and analysis the writer might require is planned and programmed in advance. In Russian contract research organisations the two must go on working together after the regulator approves the study, to develop the plan and to review the case report form, which the organisation's data management department usually generates. Medical writing review of the plan is essential for planning and outlining the report and for avoiding late changes to the statistical outputs.
Designing the statistical outputs
The statistical outputs comprise the summary tables, figures and graphs and the data listings, known in the industry as tables, figures and listings. In accounts from 2007 and 2014, the statistical team most often generated them with SAS, the industry-standard statistical-analysis software, and writers used them in their reports.
Medical writers and statisticians have a great deal to offer each other. Working with statisticians can save writers time and avoid the errors caused by retyping, copying and pasting data from statistical outputs. Statisticians can provide outputs that work within the framework of the report and evaluate whether a proposed analysis is technically feasible; writers can minimise error potential and the statistician's review time by requesting appropriate outputs, and can assist in the development and review of data analysis plans. Offering to edit the Data Safety Monitoring Board presentation and the statistical analysis plan helps the writer establish the relationship sooner, provides a useful service for the statistician, and makes the statistician more aware of the writer's data formatting requirements, which in turn makes it easier to request outputs in the form the writer needs.
One presenter listed three things a writer should ask the statistician for:
- List the top three adverse events in each treatment group, with a program that displays them as a table the writer can convert to text for the report. Many writers locate these events by scanning a table visually, which is prone to human error, and the program reduces time and errors by removing the steps of locating them and typing them into the report.
- Include the program and the data extraction date in each table. Having the statistician add the program name, that is the version of SAS used, and the date of data extraction automatically to every table reduces the risk of accidentally using older data, and the statistician can also be asked to say whenever the tables change, so the report can be updated.
- Provide a continuous table for in-text use. SAS inserts page breaks into tables by default, which can leave unsightly tables with odd gaps, and an infinite or very long table length, such as 5 miles, avoids them. Customisations can also be requested, such as leaving out the percent symbol after each number where it is not wanted, which can save time and reduce the risk of introducing an error while reformatting.
The statistician is responsible for the data and must review the writer's report to ensure they are correct, and outputs that closely reflect the writer's requirements can greatly reduce that review time by minimising the need to retype or cut and paste data.
Keeping each other informed
The statistician and the writer should keep each other in the loop. In a full-service trial at a contract research organisation the statistician is generally involved throughout while the writer is often brought back in near database lock, so the statistician may become aware of issues in trial conduct, such as delayed enrolment, early trial termination, protocol amendments, randomisation issues and protocol deviations, that could affect the report's timeline or content, and it is vital that this information reaches the writer as it becomes available. In the other direction, the writer, who is responsible for addressing reviewers' comments on the protocol and the report, needs to watch for any change that affects the statistical methods or the data analysis and should flag it immediately to the statistician for re-validation.
During the study and before database lock
In Russian contract research organisations the two remain part of the process after the protocol has been implemented at the study sites. Both may confer with the trial team and the sponsor on questions and difficulties in applying the protocol in practice, such as a high rate of premature withdrawal or loss to follow-up, difficulty in performing assessments, or problems related to data analysis or reporting, and their participation helps ensure that decisions accord with the protocol and fit the needs of the later analysis and reporting.
In the same organisations, once a study is complete and its database cleaned, the two take part in data review before database lock and before the statistical analysis starts. The step should not be underestimated, because it helps ensure the data are clean and complete for the final analysis, and a detailed preliminary review of raw data can save both people time later by avoiding problems with the final analysis and its interpretation. Their roles in it differ: the statistician performs statistical review checks to identify outliers and find missing or inconsistent data, while the writer looks for errors and inconsistencies in the coding of adverse events, medical history and concomitant medications, and for underreporting of clinical descriptions that would complicate interpretation of the collected data. Both must take part in the data review meetings where final decisions about data issues are made with the other stakeholders, and for phase II and III studies the writer is expected to work closely with the statistician on the distribution of patients into analysis sets, especially in cases requiring clinical judgement and opinion.
Reviewing the statistical outputs
The statistician delivers the statistical outputs as tables, figures and listings, and those are the primary data source for the report. The writer's review of them starts with the plan and the mock outputs and continues with the real ones, and the writer should review the draft outputs thoroughly and request any necessary revisions or additional outputs as early as possible, so that the statistician has time to deliver them without delaying the report.
The writer should review the outputs from the perspective of their end users, who include medical reviewers, investigators and regulators, and as a rule of thumb each table and listing should be a standalone document. The statistician should work with the statistical programmer and the writer to ensure the outputs meet the specifications defined in the protocol and the plan.
The success of the report depends on effective reviews of the statistical analysis plan, the final raw data and the statistical output. To keep timelines, issues with the database and the analysed data must be resolved before the output can be included in the report's core text and appendices. Substantial time can be saved when the writer reviews the prepared statistical output before starting to write, even when it is considered final after quality-control procedures, because the review lets discrepancies, errors and confusing results be identified, discussed and corrected, reducing complications and confusion during writing.
The clinical study report
The clinical study report is the writer's responsibility, but the statistician should take an active role in providing input, not only on the statistical sections but also on the results sections, with respect to the endpoints and their interpretation. The statistician accompanies the writer through the report's review cycles, ready to answer questions and clarify queries. There was a time when the biostatistician issued a separate statistical analysis report; the industry trend by 2016 was to integrate the clinical and statistical text and analyses into a single document, the clinical study report as it is now known. Once the first draft of the integrated report exists, the two go through the core text carefully together to check that the results are interpreted correctly from both a statistical and a clinical perspective. At the end of the study the two should produce an integrated report that is in effect a whole dossier containing all the trial documents they worked on, with the golden thread from the initial protocol to the final report unbroken throughout.
The last two on the study
Most members of a study team are involved from study start to last patient last visit and then move on to the next project once the database is locked. The biostatistician and the medical writer stay involved to the very end, beyond database lock, until the full report is signed off and filed in the trial master file, and only then can the two of them say the job is done.
Reading a publication with a statistician
One presenter on statistics holds that the best way to read a publication is to share it with a statistician: the writer brings the clinical point of view and the statistician the eye for the numbers, and together they make a good combination. In one such pairing, one partner found something in the numbers and the other explained it from what physicians do in the procedure; the first found it important to be aware of the clinical side, and the second found it interesting to see that the numbers reflect what happens in real life. The presenter's advice is to team up with a statistician where a writer can: it is quite fun for both, and the statistician will point out where authors have been very creative in writing publications.