04Find, Appraise & Organize the Evidence

Literature is often described as a set of papers. For the writer, it is better understood as a chain of decisions: what question is being answered, where evidence is sought, what is retrieved, what is included, how quality and relevance are judged, what each source supports, and what remains uncertain.^1,2

The chain can be brief or highly formal. A manuscript introduction may need a targeted search for disease burden, current management, and the precise evidence gap. A narrative review may require a broad and balanced exploration of a field. A systematic review treats the search, screening, and synthesis process as a scientific method that must be reproducible. The label must match the process.^2,3

Begin with the evidence need

Translate the document brief into evidence questions. A useful question identifies the topic, population or setting, intervention or exposure where relevant, comparator where relevant, outcomes or concepts of interest, time horizon, and the role the evidence will play.

Not every question needs a formal PICO framework. The discipline is to move from “find references on this topic” to “find evidence that lets us explain this specific claim, context, comparison, mechanism, burden, practice gap, or uncertainty.”

List the likely evidence functions:

• Establish the importance or burden of the problem.
• Describe current practice or standard approaches.
• Explain a mechanism, method, or conceptual model.
• Support the rationale for a study or intervention.
• Compare the new findings with prior work.
• Identify limitations, contradictions, or safety concerns.
• Support recommendations or implications at the strength permitted by the evidence.

This list becomes a search map and later a source-to-section map.

Choose the rigor before the search expands

Use a simple ladder:

1. Targeted support: A bounded search for discrete claims or background needs. Record enough detail to repeat the search reasonably, but do not represent it as comprehensive.
2. Structured narrative review: A planned search across defined concepts and sources, with documented selection principles and balanced coverage, but without claiming the full reproducibility and exhaustive methods of a systematic review.
3. Systematic review: An explicit research method with predefined eligibility, reproducible search strategies, documented screening and exclusions, appraisal, and transparent synthesis. A meta-analysis adds statistical combination when appropriate.

Convenience samples must not masquerade as systematic evidence: even in a narrative review, the literature search is a critical source of selection bias.² A systematic review, by contrast, begins with focused questions, a specified search, defined selection criteria, critical appraisal, and an explicit synthesis.¹ A writer can search carefully and still produce a narrative review. The problem is not the absence of the “systematic” label; it is claiming a rigor the process did not have.

Build search concepts, not one giant sentence

Break the question into concept blocks, synonyms, spelling variants, abbreviations, indexing terms, drug or device names, disease terms, population terms, and study-design filters where justified. Search strings should be broad enough to capture relevant language and narrow enough to remain reviewable.

Test the strategy against known relevant papers. If a seminal or clearly eligible source does not appear, inspect which concept excluded it. Avoid making the strategy so dependent on exact terminology that it misses older language, regional terms, or indexing differences.

Select databases and current search guidance according to the project. Whatever the search level, document where you looked, what you asked, when you searched, which limits you applied, and how the results were handled.^1,4

Separate discovery from inclusion

A search result is a candidate, not evidence already accepted. Maintain distinct states:

• Retrieved metadata
• Abstract available
• Full text available
• Duplicate resolved
• Screened for eligibility
• Included or excluded with reason
• Appraised for validity and relevance
• Assigned to one or more evidence functions
• Cited or used in the document

This prevents a common collapse in which a title is collected, an abstract is skimmed, and the paper enters a reference list without full-text verification. Across medical fields, a systematic review of quotation accuracy estimated a mean cited-claim error rate of 14.7%; most content errors were major mismatches in which the reference failed to substantiate, was unrelated to, or contradicted the claim.⁵

Metadata quality matters. Incorrect authors, title, year, journal, DOI, or pagination make references hard to retrieve and can propagate through citation tools. Deduplication should compare more than one field because the same work can appear through variant records.

Appraise before you summarize

Evidence appraisal asks two broad questions: can this source be trusted for the claim, and does it directly answer the question being asked?

Consider:

• Study design and whether it fits the inference.
• Population, setting, intervention, comparator, and outcome relevance.
• Sample size and precision.
• Risk of bias, confounding, missing data, and selective reporting.
• Prespecified versus exploratory status.
• Directness to the current document's population and use.
• Recency and whether later evidence changes interpretation.
• Independence, funding, and conflicts where relevant.
• Whether the full method and result are available for verification.

Do not reduce appraisal to a prestige shortcut. A highly cited paper can be indirect. A guideline can be authoritative within scope and irrelevant outside it. An observational study can answer a real-world question that a trial did not address, while remaining unable to support the same causal language. Match the inference to design, conduct, precision, and applicability—not to reputation.^1,6

Search for disconfirmation

Fair balance improves when the search explicitly looks for evidence that might change the preferred message.⁷ Search alternative terminology, contradictory findings, negative studies, adverse outcomes, limitations, and competing explanations. Citation chasing can reveal influential work missed by the first strategy, but record the path so that snowballing does not become invisible cherry picking.

The writer should be suspicious of an evidence set that is uniformly convenient. Uniformity may be real, but it deserves testing. A balanced review does not give every opinion equal weight; it represents material evidence in proportion to its quality and relevance.

Treat systematic reviews as research projects

Systematic reviews are investigations whose “subjects” are prior studies.³ Their credibility depends on explicit methods. Define the question and eligibility criteria before screening. Preserve the exact search strategies, dates, sources searched, deduplication, screening decisions, reasons for exclusion, included-study characteristics, appraisal, and synthesis method.^1,8

A meta-analysis requires statistical expertise. Pooling does not automatically solve disagreement among studies; heterogeneity, incompatible definitions, bias, and differing designs can make a single summary misleading.⁸ The medical writer should understand the analysis well enough to report it and collaborate closely with a qualified statistician or evidence-synthesis specialist.

The systematic-review burden also changes drafting. Methods must let another reader understand and, in principle, reproduce the process. Results must account for the flow of studies and the characteristics and findings of those included — the PRISMA statement's 27-item checklist and study-flow diagram exist precisely to make that reporting transparent.⁸ Discussion must separate what the synthesis shows from what the limitations of the evidence base allow.

Organize evidence for writing

A reference manager stores citations. A writing evidence system stores decisions about use. Add structured notes:

Do not write interpretive notes that are stronger than the paper. Whenever possible, retain the exact location supporting the note: page, table, figure, section, or paragraph. This reduces later source hunting.

Know when to stop

Searches expand easily because every paper contains more references and more terminology. Stop criteria depend on method. A systematic review follows its protocol and update rules. A targeted search can stop when the defined evidence needs are adequately covered, important contradictions have been investigated, and additional searching yields diminishing relevance rather than new conceptual information.

Stopping is not the same as declaring certainty. Record the search limit, coverage, and unresolved gaps. A sentence such as “No evidence was found” is much stronger than “The defined search did not identify evidence.” The latter describes what the process can actually support.

Open the systematic-review and review-article deep dive ↗