Peer-reviewing a clinical trial protocol

A clinical trial protocol is peer-reviewed internally by subject matter experts before it is finalised and submitted. The review rests on its team, a kick-off meeting, a timed review cycle, one controlled version of the protocol, briefed and trained reviewers, the resolution of their comments and a section-by-section check of the protocol's crucial elements.

What protocol peer review is and why it matters

Peer review is a process in which subject matter experts review each other's work to meet the accepted high standards of their discipline and disseminate research data, to ensure that unwarranted claims, unacceptable interpretations or personal views are not presented without prior expert review. ¹ In the peer review of a protocol, the experts consider aspects such as the study design, trial procedures, subject eligibility, feasibility, acceptability and endpoints. ²

Protocol development and peer review are vital to the clinical development programme of pharmaceutical companies and contract research organisations. ² They result in the submission of a systematically reviewed protocol to the regulatory authorities for their expert opinion and approval. ² The review team and the review process add substantial value to protocol development. ¹ Poor peer review leads to protocol amendments, which delay regulatory submission and increase project costs. ³ A thoroughly reviewed protocol can avoid significant amendments, and avoiding them speeds up the regulatory submission and saves project costs. ⁴

Peer review of a protocol is meant to: ⁵

• provide alternative wording for more clarity ⁵
• ensure a high-quality protocol ⁵
• encourage the authors to meet the accepted high standards of the therapeutic area ⁵
• ensure that the desired messages are conveyed ⁵
• ensure compliance with regulatory and ethical requirements ⁵
• assess the credibility of the study design, objectives, endpoints and feasibility ⁵
• provide comments that add value and move the protocol forward ⁵
• ensure that unwarranted claims are not included without prior expert review ⁵
• ensure that unacceptable interpretations are not included without prior expert review ⁵
• ensure that personal views are not included without prior expert review ⁵

The writer and the stakeholders

The stakeholders are responsible for the design, the scientific aspects and the regulatory, ethical and legal requirements of the protocol, while the regulatory medical writer is accountable for its consistency, accuracy, formatting and finalisation. ^{1, 2} The writer is an essential part of the protocol preparation and review team and plays a crucial role in protocol development and peer review alongside the stakeholders. ^{3, 6} The writer and the stakeholders must collaborate effectively during the review, working together to highlight the specific issues to be addressed before finalisation, which helps create effective, efficient and high-quality protocols. ^{3, 1}

The review can be inefficient and challenging for writers and reviewers when there is a communication gap between them, so effective coordination between the reviewers and the writer is essential for the review to run efficiently. ¹

Review by scientific experts is crucial for the writer to produce a high-quality protocol for regulatory submission, so the writer needs a comprehensive understanding of the peer review process and its steps. ² The stakeholders benefit from a writer who understands protocol development and the peer review process, each stakeholder's roles and responsibilities, the document management systems and the project timelines, which helps produce a high-quality document. ⁶

The review team

The team's composition can vary with the type of study and its design. ⁷ The recommended team has ten functions, and each gives its opinion on these key elements of the protocol: ^{7, 8}

• Investigator — the feasibility of the trial, trained and experienced resources, significant risks in the trial, the inclusion and exclusion criteria, operational challenges, and the benefit-risk ratio in the trial. ⁹
• Medical expert — the study design, the inclusion and exclusion criteria, the primary and secondary objectives, the endpoints, the assessment procedures, the use of concomitant therapies or the stopping rules, scientific expertise on regulatory queries and their responses, and operational challenges. ¹⁰
• Biostatistician — the statistical procedures and methods and the interpretation of the endpoints, safeguards that minimise potential variability in the study, and precautions against the various forms of bias in the protocol. ¹¹
• Data manager — the key data items to be collected and how often they are collected against the visit schedule, for developing the paper or electronic case report form. ¹²
• Clinical project manager — the description of study conduct, the feasibility of the trial, optimal execution of the study, and operational challenges. ¹¹
• Safety expert — the description of the drug surveillance programme, including medical reviews for safety reporting, safety databases, necessary follow-up, risk assessment and relatedness to the product. ¹³
• Regulatory expert — compliance with FDA and international regulations, interpretations and guidelines for designing and conducting the trial. ¹⁴
• Pharmacokinetic scientist — the description of the pharmacokinetic objectives and endpoints, dosing procedures and frequencies, pharmacokinetic requirements, and the statistical procedures for evaluating pharmacokinetic data. ¹⁵
• Formulation lead — an adequate preparation and form of the drug that stays both stable and acceptable to the patient throughout the study. ¹⁶
• Investigational product supply management lead — the description of good manufacturing practices for preparing, storing, packaging, labelling and distributing the investigational products to the study sites, ensuring that the data elements are complete and reliable, and identifying any key data element missing from the protocol. ¹²

Too many reviewers can lead to conflicts and contradictions, so the writer can propose to minimise the number of reviewers, to one subject matter expert per function. ^{17, 18} Where a function has several reviewers, the writer can request a consolidated set of comments from that function, with one contact person coordinating it. ¹⁷

The kick-off meeting

A kick-off meeting with all the stakeholders is the best way for the writer and the stakeholders to collaborate and communicate during the review. ¹ At the meeting the stakeholders come to understand the team members' roles and responsibilities, the training needs, the data sources for review, the instructions and expectations for the review, the keeping of minutes and action items, the review cycles, the timelines and the comments resolution process. ¹ The stakeholders of a global protocol may be in different places, so its kick-off meeting is usually held virtually. ¹ The writer should know the time differences between countries to achieve a robust review. ¹ The writer must consider these steps before, during and after the meeting: ¹

• Before the meeting, consider the time zones the team members are in. ¹⁹
• Confirm a meeting time with the stakeholders by email before sending the invitation. ¹⁹
• Make sure that the subject matter experts responsible for each protocol section are identified and invited. ¹⁹
• Make sure that every invitee has access to the virtual meeting platform. ¹⁹
• Have all the meeting details ready one day before the meeting: the time, link, participants' details, agenda, and the protocol synopsis, outline or concept sheet and other source documents. ^{19, 12}
• Check the meeting technology beforehand to avoid technical glitches during the meeting. ¹²
• Confirm the attendees' contact details and availability. ¹²
• Prepare a checklist before the meeting to support a productive discussion. ¹²
• During the meeting, keep a stable internet connection and clear audio, since important information can be missed when the audio fails. ²⁰
• Mute when not speaking. ²⁰
• Listen carefully to the discussion and take notes for future reference. ²⁰
• After the meeting, prepare the minutes and distribute them to all the stakeholders, to help the team with the next meetings, further actions and planning. ²¹

The review cycle and its timeline

The recommended review cycle runs in three stages over four to six weeks. ⁸

1. Drafting — the writer sends the draft protocol to the reviewers and receives their comments. ⁸
2. Addressing comments — the writer addresses the comments and schedules a meeting to address them. ⁸
3. Sign-off — the review team approves the protocol, and a sponsor representative signs the reviewed protocol. ⁸

The review can be delayed by a complex working environment and by resources that conflict with other projects. ^{22, 23} A delay can lead to a poor review because the review time is insufficient. ²³ So the writer should coordinate with all the stakeholders and aim to complete the review before the delivery date, two to three days ahead of the review timeline, so that the protocol is completed on time. ²³

Version control and document management systems

Version control of the protocol is a crucial step in the review. ²⁴ In many cases the writer receives several versions and texts from different stakeholders as email attachments or in the body of emails. ²⁴ Versions scattered this way are hard to track, their comments hard to consolidate and their issues hard to reconcile in the next draft, and the writer can miss essential comments from critical reviewers, which lowers the protocol's quality. ²⁴ So the review team should keep the protocol's versions in a common document management tool. ²⁴

Under good clinical practice (GCP), the sponsor should validate all computerised systems on a risk assessment of their intended use and of their potential to affect the protection of human subjects and the reliability of trial results. ⁵ So GCP-compliant systems are essential in the review. ⁵ Many electronic tools are available for the peer review of the protocol and other documents, and they help achieve an effective review and version control. ⁵ Potential tools include, among others: ⁵

• Veeva Vault ⁵
• PleaseReview ⁵
• Citrix Software ⁵
• a shared network directory ⁵
• Lotus Notes ⁵
• Documentum or other document management software ⁵

These systems let all the reviewers review the document at the same time and see each other's comments. ⁵ They record who checked the document out, when, and when it was checked back in, and they keep track of its versions and updates. ⁵ A system that is available and familiar to all the stakeholders is essential. ⁵

Briefing and training the reviewers

For a focused and effective review, the writer should tell the reviewers at the kick-off meeting what to focus on and how to add their comments to the protocol. ²⁵ The review should generally be strategic, focused on the content and scientific validity of the data and not on inconsistencies, numbers, spelling errors, abbreviations, language, editorial matters, style, citations, cross-references or overall formatting. ²⁵

Reviewers are in many cases unclear about what the review expects of them. ²⁶ So at the first meeting with the reviewers, the writer should clarify these expectations: ²⁶

• familiarity with the current standard operating procedures and regulatory guidelines ²³
• familiarity with the data sources: the protocol synopsis, concept sheet or outline, the current version of the investigator's brochure, the product label, the summary of product characteristics, and recent literature where applicable ²³
• instructions for the electronic tools ²³
• familiarity with the document type and its development stage ²³
• how review comments are to be categorised ¹⁷
• strategic input on content, given as specific, actionable and relevant comments ¹⁷
• a back-up plan for the review ¹⁷
• any training requirement ¹⁷
• how to respond to other reviewers' comments ²⁶
• each reviewer's focus, primarily on their own area of expertise ²⁶
• training on the electronic tools, confirming access, familiarity with the tool and any difficulties ²⁶
• the reviewers' responsibilities, the review process and the timelines ²⁶
• any other training for new reviewers ²⁶

It is essential to provide the training materials, the sponsor's and the contract research organisation's standard operating procedures, work instructions and other guidance documents necessary for protocol development. ²⁷ Every team member who reviews a protocol should receive compulsory training in: ¹⁸

• how to review documents ¹⁸
• how to give comments that are content-related and actionable, not editorial or stylistic ¹⁸
• how to prioritise comments as critical, major or minor ²⁸

Making and resolving comments

All the reviewers are experts who provide scientific comments to the writer. ²⁹ They should make their changes in track-change mode together with comment boxes, which is more helpful and efficient for the writer. ²⁹ The success of the review depends on each reviewer focusing on their own area of expertise and trusting their teammates to focus on theirs. ²⁹

The best way to resolve the comments is a comments resolution meeting with all the stakeholders, where differences of opinion should be discussed openly until a consensus is reached. ¹⁸

Crucial elements to check, section by section

For an effective review, reviewers should consider the crucial elements of each protocol section in the following table, which help develop a high-quality protocol. ⁷

Section	Crucial elements for peer review
Introduction	The current prevalence and incidence of the disease; the rationale for the choice of study design elements; the goals of doing this particular study at this point; the research question and the justification for the trial, including a summary of relevant studies; an unmet medical need for any indication; known and potential risks and benefits; an explanation of the choice of comparators; drug- and disease-specific background information, including the safety information available; competing products on the market
Objectives	The question or questions the study is designed to answer; whether the trial is planned for superiority, noninferiority or exploration, and the scientific rationale for it
Evaluations and endpoints	Whether the trial needs to be conducted at this stage; whether it needs to be combined with another trial, if applicable; whether the endpoints support the objectives; whether the endpoints are clinically and scientifically valid for the disease studied; whether the estimands are clearly indicated; whether the endpoints chosen are the best ones to measure; the tools, instruments or questionnaires and laboratory tests that will gather the data for the efficacy endpoints; the evaluations required for the primary and secondary endpoints; the references for the development and validation of the instrument's content; the patient population in which the questionnaire was validated, with special attention to the current study population; the specific time points and their acceptability to the regulatory authority, such as change from baseline to a given week; consideration of including definitions of the derivation, use and timing of a composite endpoint
Hypothesis	The types of hypothesis used in the trial and the reasons for choosing them; whether a hypothesis is stated in the protocol, and if not, a convincing reason in the protocol for not stating one; statements of the hypothesis that consider the endpoints studied, including when they are measured, such as the day, week or visit; a hypothesis that is not a rewording of the objectives; careful review of the study hypothesis, whose verification is a very important aspect of the study
Study design	Whether the design is the best one for the trial, why, and whether other designs were considered; whether it can control the major sources of bias; what is done to minimise the placebo response; feasibility for patients and doctors; whether the trial has to be randomised, and if so why; the method of assigning treatment to subjects, such as randomisation, or other measures to minimise bias, such as stratification variables; the level of blinding, such as open-label or double-blind; competition for the trial, for the patient population, for institutions and for industry trials; any specific study setting required, such as a community clinic or a tertiary care hospital; the sequence and duration of the study phases or periods, including any follow-up phase, and the expected duration of subject participation; the end-of-study definition; for an active control, whether the intent is to establish superiority, noninferiority or equivalence of the study drug against it; the rationale for the study population, level of blinding, treatment groups, dosage and dosing interval, route of administration, treatment period, control, efficacy measures and the length of study phases and periods
Time and events schedule	The efficacy and safety parameters and the pharmacokinetic, pharmacodynamic, biomarker, pharmacogenomic, immunogenicity and other measurements, with their frequency and timing in the schedule; whether any procedures can be eliminated or reduced in number or frequency; whether any patient visits can be eliminated; whether any tests at a visit can be eliminated; whether the protocol lists what is to be done at each visit; whether the patient visits and assessments match the table of assessments
Eligibility criteria	Whether the criteria are necessary for the trial; whether they are reasonable, too restrictive and could be relaxed, or not restrictive enough; whether some patients with other health problems, such as diabetes, could be eligible; what a life-expectancy criterion is based on, and whether it is necessary; whether any inclusion criteria can be eliminated; whether the inclusion criteria will create the most appropriate group of patients for extrapolating the data; whether the inclusion criteria are realistic for patient recruitment
Patient population	Whether the population studied is the most relevant one to meet the company's goals; whether it has appropriate gender and minority representation; whether it contains elderly patients, and whether it should
Blinding, if applicable	Whether the issue is adequately addressed; whether all the groups that should be blind are, such as those who interact with the primary investigator, the patients, site staff and the sponsor; whether the protocol adequately deals with blinding the drug container and packaging labels and with how to unblind patients in case of problems
Concomitant therapy	Whether the protocol deals appropriately and adequately with the issue; whether it lists acceptable and unacceptable prescription drugs, over-the-counter drugs and other nonprescription products and the terms under which each may be used
Patient compliance	Whether patient compliance is monitored or measured in the trial, and if so how; whether that is the best way, and whether other ways were considered
Safety reporting	Compliance with country-specific regulatory requirements for safety reporting to the regulatory authority, institutional review boards or independent ethics committees, and investigators
Pregnancy reporting	Compliance in collecting and reporting pregnancy information, including abnormal pregnancy outcomes such as spontaneous abortion, fetal death, stillbirth, congenital anomalies and ectopic pregnancy
Contraception guidelines	Contraceptive use consistent with local regulations on the methods of contraception for clinical study participants
Data protection	Whether the protocol includes a data protection section; compliance with the applicable rules on the protection of personal data, and any relevant information on the measures to take after a data security breach
Start and end of study	Whether the protocol includes a clear end-of-study definition; whether it includes a clear study completion definition; whether the two definitions differ under the regulatory requirements
Stopping criteria	A description of the stopping rules or discontinuation criteria for individual subjects, for study periods and for the entire trial
Compliance with ethical and regulatory requirements	Whether the protocol is designed and developed in compliance with the applicable ethical and regulatory requirements; where applicable, whether it follows guidance documents for its indication or therapeutic area

^{30, 31, 32}