Judge a review process by whether it is defined, resourced and enforced
Whether a document review among several reviewers works depends less on the reviewing tool than on the process around it: a written process, the tool used as intended, training in what reviewing means, staged reviews, reviewing planned as a defined activity, and enforcement. A review also takes a route, by files circulated by email, in series or in parallel, or in one shared document, and each route changes how much of the review the writer controls.
Reviewing tools and the practice around them
Collaborative reviewing tools for regulatory documents were available by 1999, yet in 2015, in one contractor's experience, their use remained an exception. Such a tool promised a secure drafting environment, guaranteed version control, workflows running reviews in a structured process within defined timeframes, reviewers able to see and respond to each other's comments during the review, automated collation, reminders of reviewers' obligations and an audit trail of reviewing activity. It was to end distribution by email, comments arriving in multiple documents or in permutations layered on other people's comments, reviews of the wrong version, and medical writers consolidating comments into a single file by hand. Adding a few other ingredients to such a tool, such as training the team in reviewing expectations and some coordinated planning, gives a genuine increase in the efficiency of document review: documents are reviewed more effectively, quality goes up, and time and cost go down. At one company that had the tool in 1999, the workflow function had been disabled, so the separate files of review comments and the painful and costly manual consolidation continued unabated, though a functional document repository was at least in place.
By 2015, apparently very little had changed at many companies, if not most. Among the clients of one contracting company, which ranged from global multinationals to biotechnology start-ups, well-planned and technologically advanced document production was the exception, and poorly planned and technologically challenged production tended to be the rule. Of the around 100 clients of all sizes it worked with between 1999 and 2015, only five used collaborative reviewing tools as intended, and only three of the five combined the tool with a well-structured process. Many possessed the software without using its full capability, in effect the same situation as in 1999. Document review also needs effective management and planning of resources, which are frequently lacking, and medical writers are often left preparing documents reviewed under less than optimal procedures even where software solutions are already available or could be had for little cost. Good organisation and management will probably always remain a challenge, implemented with varying degrees of success, and the collaborative reviewing tools available since 1999 have not been fully embraced across the pharmaceutical industry. The contractor holds that six ingredients are vital to effective document review, all of which the three leading clients use effectively.
A defined review process
The first ingredient is a defined, structured review process, which may sound obvious but is surprisingly often partly or completely absent. In a highly regulated environment the obvious form is a standard operating procedure or other written guidance detailing the process step by step, with procedures defined for what is to be reviewed, how the review is to be conducted, when, and by whom. These procedures should be pragmatic and non-negotiable. There is nothing new or mysterious about the management and planning skills effective document review needs. Where a structured review process is lacking, confusion within the reviewing team is almost guaranteed, diverse approaches to reviewing can result, and obtaining the consensus needed to move documents forward can be challenging for the writer. As part of a structured review process, a project's kickoff meeting should also clarify which timelines are realistic, where the interdependencies and potential threats lie, and how those threats would be mitigated. All stakeholders, contractors included, then need to agree and commit to the timelines. Ideally the timelines are then actively managed through the project, with agreement and commitment obtained from all stakeholders whenever the original plan has to change.
The reviewing tool
The second ingredient is a collaborative reviewing tool, which, used correctly, increases the effectiveness of document review dramatically. The greatest obstacle to implementing one is often resistance to change: some clients have insisted such tools are impossible to use while others have used them successfully for years, which makes it a question of enforcement. Implementation cost is another perceived obstacle. It arises not because a company cannot afford the software, since any company that can afford to develop drugs can afford the resources for effective planning and a collaborative reviewing tool, but because the cost has not been justified in the right places, perhaps made worse by the resistance to change. Nobody can seriously claim it is more cost-effective to have medical writers manually collate comments from multiple files than to have the software do it automatically at a medical writing cost of almost zero. Training people to use the tool is a further challenge, especially where external collaborators may also have technical difficulty accessing the software, but such challenges are anything but insurmountable.
Training in what reviewing means
The third ingredient is training in what reviewing means, which receives remarkably little attention. Everyone involved in writing and reviewing documents should receive mandatory training in how to review, covering how to prioritise comments, for example as major or critical ones that must be addressed, minor ones the team can address at its discretion, and cosmetic ones; how to respond to other reviewers' comments where a collaborative tool is in use; and how reviewers should focus primarily on their own area of expertise. Another major role of the training is to impress on reviewers the need for constructive, specific and unambiguous comments. A good understanding of reviewing expectations is also needed for resolution meetings.
Staged reviews
The fourth ingredient is staged reviews. To reduce the writing and reviewing burden in the later stages of document preparation, when time will probably be tight, it can help to plan a staged writing and reviewing process. In a study report, data-independent sections can usually be written and reviewed before the data become available, the aim being consensus on those sections without the complication of thinking through the results at the same time, and the process can also help with structuring and reporting the results. Teams usually find staged reviews attractive as a promising way of meeting timelines and readily agree to them, often without fully comprehending the consequences. The review of those sections really does have to happen in the earlier slot allotted to it, and true commitments about structure and messaging have to be made then. If the review takes place as planned, the reviewed sections should be locked down on the clear understanding that they will not be revisited and reviewed again later. Changing the colour of the text and instructing the team accordingly can lock them down, and where a structured review process is in place and everyone understands the processes to be followed, this should come as no surprise to the team.
Reviewing as a planned activity
The fifth ingredient is reviewing planned as a defined activity. Reviewing documents really does take time, however reluctant managers are to understand it, so it is essential to plan reviewing with realistic and negotiated timelines, including the resolution meetings, which reduces the risk of an excess of competing priorities, with reviewers having to review several complex documents simultaneously. One effective approach, practised only rarely, is scheduling reviews in electronic calendars as all-day appointments for the intended review period: the appointments do not block reviewers' calendars or prevent further reviewing requests or workflows being issued, but they make the reviewing burden at any given time visible. Electronic calendars should also generate automated advance notices of impending reviews and reminders during them. Where timelines are extremely tight, blocking a particular day for a review, as an activity that excludes competing ones, can be effective. Another essential component is a defined follow-up procedure for stakeholders who do not complete their review as planned. Ideally the process lets reviewers review in the allotted period by minimising competing priorities. Even so, a culture should prevail from the outset in which failing to respect reviewing timelines is inconsiderate to other team members and unacceptable. How far this is enforced varies greatly across companies, and the leading ones enforce it vigorously. Even under the best conditions, circumstances will always arise in which a reviewer cannot deliver on time, and the response depends on how essential that reviewer's input is. Some companies operate a policy that no response is agreement and move on, which can be problematic when senior management input is needed, but even with such influential reviewers an agreed strategy for non-responders is needed to let the team advance the document to the next stage.
Enforcement
The sixth ingredient is enforcement. Companies often have some form of guidance in place, sometimes a standard operating procedure, that is not rigorously applied, and even a defined review process available in writing often goes unenforced, in part or even completely, which can be due to inadequate project leadership or an aspect of the corporate culture. The few effective review processes one contractor has witnessed were effective because the process had, and was seen to have, management authority at the highest level. Initial and refresher training events are needed to reinforce reviewing principles and processes, together with management's expectation that the defined procedures are mandatory, so that a culture can develop in which it is absolutely clear that the reviewing process is not negotiable. The training must also reinforce that any delay in providing review comments complicates the writing of subsequent drafts, can delay timelines, and disadvantages other stakeholders in the project. Almost every medical writer can recount projects where a lack of reviewing discipline, especially comments provided late, has been a substantial challenge. However experienced a medical writer is, document review processes are a shared pain: every medical writer has at least one horror story about review cycles going awry, not because writers are precious about their work but because of the havoc and heartache that poor reviewing practices leave in their wake. In companies where good reviewing practices and discipline are lacking there is often resignation and indifference, with muddling along pervading as an acceptable corporate culture in the absence of any empowerment to change it.
Many medical writers are, for any number of reasons, still writing documents within ineffective review processes. Medical writers are in many respects best positioned to advise on making document review more efficient, and in some ways they also partly contribute to the problem, because they often act as a buffer that lets teams meet their timelines despite all the inefficiencies along the way. The classic illustration is how writers react to delayed comments. On one 2015 project the client, with no negotiation, suddenly cut the timeline by four weeks so that the review cycles had to be compressed. A key reviewer then delivered no comments by the planned deadline, returned them in five separate files staggered over the following two weeks, and did not review the entire document, leaving the missing parts for the next draft and making that later review more complex than planned. The last file arrived a couple of days before the next draft was due, leaving no opportunity for team resolution of the issues the writing team could not settle, so these too had to be saved for the next draft. Most of the vital ingredients were lacking at that client. Some clients, asked why things are as they are, answer that certain of these practices are unenforceable at their company, although the contractor has seen exactly those practices enforced and working well at other companies.
Working with challenging reviewers: prepare, explore, engage
Problems with reviewers range from being late to meetings to breaking timelines, and one communicator offers three basic steps to help train reviewers to be better team players: prepare, explore and engage.
Thorough and intelligent planning can often ease many of the issues that arise with reviewers over a project, and preparation before the project kickoff meeting is an excellent place to start.
- Obtain a list of meeting attendees and identify their roles, noting especially the decision makers.
- Obtain or build a timeline, and check the team calendar for "showstopper" days, such as holidays and planned vacations, that might delay it.
- Lay out a plan for escalation.
- Highlight the parts of the project that need the most reviewer attention.
Preparing also means anticipating issues that may arise with particular reviewers, and directing a reviewer's attention towards the sections that need addressing, such as the data, and away from sections already finalised can save valuable time in review.
Two main exploratory techniques can be used to keep reviewers on schedule and cooperative: finding a champion and establishing a process. A champion is "a mentor—someone who can offer advice when you need it", who can potentially lobby for the writer, the goal being someone who can help the writer without changing the team dynamic. A process for reviewers increases consistency and reduces redundancy, and a process of support for reviewers may help improve the relationship between writer and reviewer.
Engaging a challenging reviewer means reframing the relationship as engaging a problem child rather than being in conflict with one, and communication is essential: misunderstandings, questions and conflicts need to be addressed with the reviewer, which can be done throughout the process, including preemptively. The occasions for it are right after the kickoff meeting, with a follow-up email reaffirming the timelines and commitments made there; when there are miscommunications; when deadlines or pieces of work are missed; and when the audience and message have been lost in the narrative. When a reviewer misses a deadline, "the circle back" means going back to that team member to talk about what is missing, what the writer needed from them, or what they did not do. A difficult reviewer can be retrained to participate effectively: by being clear, respecting the writer's own part in the problem, having a plan and knowing when to escalate, the writer can help the reviewer become a helpful resource, enriching both the product and the experience. Nothing will change in working with reviewers unless the writer changes it.
The route a review takes
However much of the writing a medical writer does on a regulatory document, they will need to take it through at least one review cycle with multiple reviewers before sign-off can happen. Moving a review from circulated files to a document edited in place changes how much of the review the writer controls.
In the traditional cycle the document is sent out to the reviewers, who comment and make changes, usually in tracked-changes mode, and return it to the writer, who collates and resolves the comments and may need to arrange an adjudication meeting for particularly contentious comments or for contradictory comments on the same part of the text. Reviewers often copy the other reviewers so that later ones can add to what is already there, which saves collating work and lets some early comments be addressed by later reviewers, leaving fewer issues to adjudicate, although such serial reviews can produce messy, difficult-to-read documents. The writer still receives comments in many versions, and unifying them in one document can be time-consuming, but the writer keeps considerable control over the document, or ownership, as some call it.
Establishing an efficient editing process is critical in collaborative writing, and a circulated draft can go to the team in parallel or in series. In in-parallel editing the leader sends the document to every team member at once and then incorporates all the revisions into one document. Keeping track of each author's edits is then challenging for the leader, members may make duplicate edits or edit an old draft, and each member cannot see the other authors' perspectives and may not be updated on the status of the project, and one speaker on collaborative writing discourages in-parallel editing for these reasons. In-series editing is a time- and effort-saving alternative: the leader sends the draft to the first author, who passes the edited draft to the second, and so on until every author has edited it and it returns to the leader. This eliminates the possibility of duplicate edits and lets each author see the revisions the others suggest.
With a web-based collaborative review application, the document is stored centrally, safely behind the company firewall where that is appropriate, and the writer sends the reviewers, once granted the necessary permissions, a link to it. The reviewers comment and make changes directly on the central document, which such applications let them edit in series or at the same time, and they can see each other's comments and changes. Because the edits are all in one place, the writer saves time and can in theory spend more time resolving issues.
Shared review has potential downsides. With email rounds the writer can filter out extraneous noise and keep discussion focused, but with a shared document the writer has less control over the discussion, which leaves room for needless bickering. Version control also becomes harder: with email rounds, a writer who diligently archives the reviewed documents has a ready record of who did what and should be able to identify where a mistake or an inappropriate edit occurred, which a shared document makes more difficult. A reviewer who accepts changes, often just to make a document edited by several reviewers more readable, may erase a change forever. Regular backups may go some way towards a detailed record of changes but may not capture all of them, and backups visible to all reviewers, for the sake of transparency, but not appropriately labelled bring a new danger that reviewers edit the wrong document. Collaborative editing systems can make the work easier, but the processes involved need careful thought, and the writer would be well advised to make an effort to retain ownership of the document to help keep everyone focused.