Designing a study of the profession
Research into medical writing starts with an answerable question, then chooses observations, measurements, samples and methods that fit it.
Research questions and observations
Regulatory, instructional and marketing communication may have distinct challenges and research needs that require separate investigation.
The research question determines who or what is studied, the methods, scope and possible uses of the results; it should be meaningful, focused, accurate and answerable. Reflecting on the previous year’s work can generate questions about whether the value produced covered salary, what evidence could establish that, and how greater efficiency or effectiveness could be recognised.
Units of observation can be parts of documents, such as introductions or figures, complete documents, individual writers or editors, writing groups or whole organisations, each with different practical challenges. An approximate answer to an important question is preferable to an exact answer to a trivial one, so ease of investigation should not determine what is studied.
Measurements
Research needs defined inputs, outputs, characteristics and explanatory and response variables. Objective measurements assign the same value when anyone follows the prescribed rules, as with counts of accepted articles or typographical errors and time taken to summarise a document. Unobtrusive measurements are particularly useful because they do not intrude into the setting: counting newsletter copies discarded shortly after distribution, for example, provided an indication of their value to readers.
Subjective measurements capture judgments or opinions, such as proposal quality, title effectiveness or personnel performance. A connotative definition can make an intangible quality more concrete: report quality might be judged by whether a draft is suitable for the chief executive, division chair or laboratory manager, representing higher, intermediate and lower quality. Judges may still disagree but are at least answering the same general question.
The unit of observation also affects the number of judges needed: estimating passive-sentence frequency in abstracts probably calls for a few trained judges and many abstracts, whereas evaluating a slide set’s quality probably calls for many judges.
Research without measurement requires careful interpretation. Qualitative research, originating in anthropological participant observation, involves detailed observation of people or events in natural settings and can use open-ended questions and follow-up to characterise difficult concepts or subtle interactions. Its findings may not generalise to other settings but often generate hypotheses and questions for quantitative investigation.
Samples
A sample should ideally be large and representative of the population to which results will be generalised; size may be set by a power calculation or by available documents or writers in a particular period or group. Representativeness is difficult because authors, editors and writers differ in skill: better ratings for manuscripts edited by one group may not apply to another.
The sample must match the question, and scientific expertise does not necessarily imply scientific-communication expertise, even with extensive publication experience. Counting subtitles in published article titles describes practice rather than establishing their effectiveness; a focus group of communication experts or an experiment on title effectiveness may answer that question better.
Research designs and data sources
The research question may determine the research design.
Literature reviews can answer or approach some questions and provide background for any study, particularly by identifying theories and findings applicable to writing and events in medical-writing history. They may also reveal organisations with related interests or alternative indexing terms for the subject.
Archival and database research can use regulatory submissions, published journals, evaluation reports and other institutional records. Reviewing print-shop invoices at the Cleveland Clinic, for example, estimated spending on patient education materials; a total below spending on business cards supported an argument for improving the materials’ quality and quantity.
Surveys can gather extensive information quickly, conveniently and inexpensively by email, web or paper, and repeated surveys can identify trends. Questions should be pretested for consistent understanding and complete, appropriate response options, and approval obtained before administration. A clinic survey found most authors would accept a one-week editing turnaround but little longer.
Reader interviews and focus groups are useful for assessing communication quality and acceptability, although conducting them requires skill. They can produce unexpected insights into perceptions of documents or people and suggest procedures, workflow changes or further research.
Experimental designs support formal comparison and hypothesis testing, reducing or controlling error, confounding and bias while examining differences between, for example, two consent-form versions. Comparisons can be between groups receiving different versions or within a group tested twice, such as before and after editing.
Team, time and resources
A research team should be assembled with attention to its size, members’ research quality and their ability to work on the project after hours. The team should agree authorship and record the consensus before developing a budget that can include advertising, a statistician and slide preparation.
Ideas for an important research topic can be generated by:
- Attending conferences.
- Reading widely.
- Recording ideas on a whiteboard.
- Letting the mind drift occasionally.
- Reviewing and prioritising the ideas.
Dividing the chosen topic into manageable parts, with each team member responsible for one, can make research fit around other work. An independent statistician should review the data before the team does, and feedback on the project should be sought before journal submission.