10Use AI Without Handing Over Judgment

Generative AI can support structured authoring, automated narratives, literature review, content reuse, and quality checks, while introducing risks involving confidentiality, fabricated content, bias, inconsistency, and misplaced trust.^1,2,3 AI can accelerate bounded parts of the workflow, but the writer remains responsible for whether the document is true, fit for purpose, and safe to release.

Start with the document task and workflow

Define what work you want to improve. Suitable candidates may include formatting references or abbreviation lists, converting images to editable text, preparing slide scripts, creating search strings, writing macros, basic data exploration, content mapping, structured reuse, drafting from defined inputs, and assisting quality checks.^1,3

Classify tasks by consequence:

• Low-risk transformation: Format, classify, deduplicate, compare, or convert material where errors are easy to detect.
• Assisted production: Propose an outline, summary, paragraph, script, query, or visual from specified sources, with mandatory review.
• High-risk judgment: Decide evidence quality, interpret ambiguous statistics, resolve conflicting guidance, assign authorship, determine regulatory acceptability, or approve final claims.

AI is most defensible when the task is bounded, the input is controlled, and the output can be checked. High-risk judgment should remain with qualified people, even when AI organizes the material they inspect.

Understand probabilistic output

Deterministic expert systems are predictable and template-like, whereas language models generate probable continuations from learned statistical patterns.^1,3 A language model does not understand a study as a human investigator does, and it may not be able to identify the source behind a statement. It can produce fluent falsehoods, invented citations, outdated practices, biased framing, or inconsistent answers to the same instruction.

Fluency is therefore a risk factor. Awkward output invites scrutiny; polished output can pass unnoticed. Treat generated content as a proposal whose confidence is unknown until verified.

Retrieval-augmented workflows can provide relevant source snippets to a model and may allow citations, but retrieval does not guarantee that the right passage was selected or that the generation represents it correctly. The evidence still needs human inspection.

Protect data before adding project material

Permission, employer and client policies, nonpublic content, proprietary information, personal data, and protected health information must be resolved before project material enters an AI-enabled workflow.^1,3,4 Determine:

• Whether the content is public, confidential, personal, proprietary, or regulated.
• Whether the client, employer, publisher, or institution permits the use.
• Whether inputs are retained, used for training, exposed to vendors, or capable of leaking.
• Where data are processed and stored.
• Who can access project instructions and outputs.
• Whether the workflow and contract meet the project's security requirements.

Removing a name does not automatically anonymize clinical information. Patient narratives and small subgroups can remain identifiable through combinations of facts. If the permission or system boundary is unclear, do not upload the content.

Set the production contract before drafting

Useful project instructions resemble useful section instructions. Specify:

• Role and task
• Approved source set
• Audience
• Output type and length
• Required content
• Prohibited inference
• Terminology and style constraints
• How to show citations or source locations
• How to mark uncertainty, missing information, and conflicting evidence
• Expected format

The PLANTS reminder captures six useful constraints: persona, length, audience, nuance, type, and style guide.¹ The deeper principle is constraint. An unconstrained request asks the system to invent both the task and the answer. A production contract reduces ambiguity and makes review possible.

Require visible provenance

Ask the system to identify which source supports each material statement and to quote or point to the supporting location for reviewer inspection. Do not allow it to fill gaps with material outside the defined dossier when the task is meant to use a controlled evidence set.

If no source supports a requested claim, the correct output is a gap, not a plausible sentence. Design the workflow to reward abstention. “Not found in the supplied sources” is more valuable than a fluent invention.

Verify by risk, not by convenience

Check generated output for:

• Factual and numerical accuracy
• Source match and citation existence
• Population, design, time point, and analysis status
• Missing qualifiers and limitations
• Unsupported causal or comparative language
• Omitted contradictory or safety information
• Hallucinated structure, requirement, or terminology
• Confidentiality and privacy leakage
• Bias and inappropriate generalization
• Style consistency and accessibility

Do not ask the same model to be the sole verifier of its own work. Automated checks can help identify candidates, but material claims need independent comparison with the source.

Keep gates human

Human decision points should include:

• Accepting the project frame and evidence boundary
• Choosing the governing stack
• Accepting the Structure
• Approving key messages and limitations
• Resolving source conflicts
• Accepting or rejecting substantive generated changes
• Confirming statistical and specialist interpretation
• Approving final release

Removing writers from the workflow would also remove document leadership: the work of breaking down tasks, setting timelines, critically evaluating sources, coordinating expertise, and gaining consensus.^1,3 That integrative work is central to medical writing.

Evaluate the workflow against the use case

Assess the underlying model or system, modifications, data access, retrieval, determinism, security, output consistency, auditability, and ability to meet the task's risk profile. Confirm that the workflow preserves authorized sources, visible provenance, controlled access, reviewability, and the handoffs required to produce the document safely.

Pilot one or two narrow use cases. Record what works, what fails, what instructions and examples improve consistency, how much review time remains, and whether the workflow actually reduces total effort. Include the time needed for training, validation, governance, and correction.

Make use transparent

Follow applicable client, employer, journal, and institutional expectations for disclosure. Maintain an internal record where the quality system requires it: system, task, source boundary, output, reviewer, decision, and version. Transparency does not mean publishing every project instruction in every context; it means that material assistance is not hidden from the people accountable for the document.

Open the AI governance deep dive ↗