Reviewing a manuscript for a journal
A peer reviewer owes the journal and the authors a competent, honest, confidential and timely assessment that answers the questions journals put to every reviewer, and does it with little training and little in return. A careless or flippant review hurts the author who receives it, and publishers asked reviewers to keep the confidential manuscript out of AI tools, and some journals asked them to declare any use they made of them.
What a reviewer owes
Someone who wants to become a peer reviewer can send the journal editor a letter and a curriculum vitae, volunteering to serve and stating the areas in which they have expertise and are willing to review papers; the editor will reply and perhaps add them to the review team.
Editors want reviewers who are knowledgeable in the topic, intellectually honest and time-sensitive, since the author and the editor cannot wait six months for a paper to be reviewed.
A reviewer of a report of a clinical study should know research methodology and basic statistical analysis.
A peer reviewer's duties are:
- Accept a paper to review only if the job can be completed within the time specified in the editor's request.
- Agree to referee papers only in your own areas of expertise.
- Maintain confidentiality about the paper.
- Disclose any possible conflict of interest, and decline a review if there is any potential difficulty.
- Write a thoughtful review that is honest and free of bias.
- Aim to make the paper the best it can be, balancing criticism with suggestions for improvement.
- Avoid excessively harsh comments, especially those that could be read as a personal attack on the authors; the opportunity to comment anonymously on unpublished papers can bring out sadistic traits, which must be recognised and stifled.
- Submit the review promptly, because the authors are anxiously awaiting a verdict.
Reviewers and editors must give a manuscript a conscientious reading in a timely manner, usually 30 to 60 days.
Their comments to the author must be constructive about how to revise, because derogatory and dismissive comments make neither the manuscript nor the researcher better and contribute little but a hostile environment.
They must keep the review process confidential, never discussing the nature or number of revisions or the comments and corrections offered, and keep the information in the manuscript confidential, neither using nor sharing it until it is published.
They must report possible ethical violations, such as plagiarism, fabrication and duplicate submission.
The peer reviewer never contacts the author directly.
The reviewer helps to improve the paper, making it clearer, more informative and often shorter, even if the journal ultimately rejects it, since the paper will then be revised and submitted to the next journal on the author's list.
One clinician who writes on getting published thinks of the roles of a peer reviewer and a practising clinician as somewhat alike: both are expected to behave ethically and to be committed to high-quality service, both should be knowledgeable, capable and thorough, both need to examine details while keeping a broad perspective, both must be reliable and trustworthy and believe that they serve a worthy purpose, and if careless, both can cause a lot of harm.
What a review answers
Reviewers are asked to answer a list of questions that is much the same from one journal to another: whether the science appears to have been done well, with the question well framed and the methods and analysis appropriate; whether the data support the conclusions, and whether any speculation has been identified as such; whether the observations are novel, or, if merely confirmatory of something already published, whether the confirmation is so important that it warrants a separate publication; whether the observations are important, moving science forward in a useful direction; whether the paper fits the scope and the audience of the journal; and whether the journal ought to publish it, and if so with what priority.
A few journals add a question or two, such as whether the paper can be made shorter or clearer, and whether the reviewer spotted any ethical concerns, including whether the use of animals and of human research subjects was appropriate.
Learning to review
How to be a good peer reviewer is a neglected topic, as the authors of one guide to writing for peer-reviewed journals note: for one writer, and no doubt many others, the only instruction on starting out was "Have a look at this manuscript".
Pat Thomson and Barbara Kamler's Writing for Peer Reviewed Journals closes with advice on peer reviewing: sets of questions a reviewer should ask while reading a manuscript, and rules to follow when writing the review.
Philippa Benson and Susan Silver's What Editors Want gives a chapter to who does what in peer review, also a neglected topic, presenting the different types of peer review and outlining some of the benefits of being a peer reviewer, while only briefly highlighting some of the potential biases and prejudices of editors and reviewers, a topic that in its reviewer's view warrants more detailed analysis.
Recognition for reviewers
Many peer reviewers are senior academicians and investigators, and all are volunteers who do a lot of work for no pay.
When Mayo Clinic Proceedings asked its authors, reviewers and editorial board members, in January and February 2009, how reviewers might be compensated other than with money, respondents' suggestions included formal documentation of participation that a reviewer could share with a department head, and a reviewer ranking.
Of the 105 respondents who answered the open question about recognising reviewers, 39 (37%) had no suggestion or thought the existing acknowledgment adequate, a list of reviewers' names the journal published annually in its December issue; the study's table puts this share at 38%, and its discussion at about 40%.
The other suggestions were continuing medical education credit for reviewers who complete high-quality reviews (about 18%), a document or letter a reviewer could share with a department head or supervisor to document the contribution (11%), free materials with the journal's logo such as subscriptions or pens (7%), a rating or ranking distinguishing repeat or high-quality reviewers (5%), an invitation to high-quality reviewers to join the editorial board (2%), a thank-you party or reception (2%), and publishing the list of reviewers on the journal's website (1%).
About 5% of respondents said no acknowledgment was necessary, because being a reviewer is an expected part of the academic process.
Asked about the frequency of invitations to review, 75% of 342 respondents called the rate adequate, 23% wanted to be invited more often and 2% less often.
With roughly 500 reviewers a year, the editorial office judged that respondents' suggestions for other forms of recognition may not be economically feasible.
The editorial office concluded that its current reviewer compensation practices could be enhanced.
After the survey, the office made several changes for reviewers, some of which coincided with upgrades to its manuscript system: a list of current reviewers is now sent monthly to the circulation manager, who activates a free online subscription, as some respondents had requested; the thank-you email to reviewers was revised to tell them of that subscription and to carry the journal's current metrics; printable letterhead was made available and is supplied to reviewers on request; and a cover-letter button gives reviewers quicker access to the authors' cover letter.
Humour in a review
One endocrinologist, from an experience of his own, advises referees to avoid all attempts at humour, because when an article is criticised, its author feels pain of varying intensity and duration, however senior or junior.
He had been certain that an American colleague had refereed one of his articles for a specialist journal, since the style, the manner of expression and the content of the argument all pointed to him, and, relaxed at a conference dinner, asked the colleague why he had hated the work with such intensity.
The colleague said he had never refereed any of the endocrinologist's manuscripts; he had, however, recently had an article of his own rejected on the endocrinologist's recommendation, and the journal had sent him the referee's comments with the endocrinologist's name on them.
The endocrinologist had once identified referees by the typeset, an unhelpful talent in the computer era, and had told himself he could identify one by the style and language of the review; the episode shattered that illusion.
Two or three months later the colleague sent a fax saying he had reviewed his refereeing records and found that he had refereed two of the endocrinologist's articles, including the one in question; the endocrinologist was deeply impressed, reading the correction as driven by conscience.
Reviewers, editors and AI tools
In 2024 publishers asked peer reviewers to keep manuscripts out of AI tools. JAMA asked its peer reviewers not to put confidential manuscripts into AI models, because they are confidential. Springer Nature asked reviewers not to upload manuscripts into generative AI tools, and had an internal tool that can detect whether that was done. Taylor & Francis told its in-house editors, its academic editors and its peer reviewers alike not to put content into a third-party platform.
JAMA asked reviewers the same question it asks authors, whether they had used AI to help with the review in any way, and about 0.6% said they had. Most used it for language, the journal having many reviewers whose first language is not English, or to find out whether the statistical test or the design was the right one for the study being reported. JAMA's editor was in no doubt that reviewers were using AI, and the journal hoped they would say how, so that it could guide them towards good use of the tools.
At JAMA, two authors whose manuscripts were not accepted appealed because a reviewer had used AI. In the first case, the author appealed on the grounds that the second reviewer had used AI to write the review, a complaint carrying the common misunderstanding that reviewers make decisions, when editors do. The reviewer had not declared it; asked, they replied at once and apologetically that they were early career, from a country where English is not the first language, honoured to have been invited, and had put their own review rather than the manuscript into the tool to improve its language. What gave it away was formatting: the author had noticed double asterisks wherever quotation marks appeared. JAMA's editor called this use fine. In the second case, an author appealed because a reviewer had said they used a model, and the review itself said what for, assessing whether the statistical technique suited that type of study, followed by a bulleted list. The reviewer had told both the journal and the author, and the use was not why the paper was rejected.
One author said in 2024 that, especially over the preceding year, they had received many reviews with a machine evidently behind them and no declaration of it. JAMA was not using AI to conduct peer review in 2024. Its editor had heard of a few journals that did use AI, at least one in the United Kingdom using it to draft decision letters, but could not speak for the thousands of journals in the world.
The Lancet did not want generative AI used for peer review, because in its judgement AI could not assess conceptual and scientific advances to the degree required, though that might change. The BMJ was running peer-review pilots in some of its journals. Elsevier, The Lancet's parent company, had made a generative AI tool available to editors in the preceding couple of months; it was a closed platform and therefore confidential, confidentiality being part of the worry about peer reviewers using AI. The Lancet was still working out how it wanted editors to use the tool, few were using it much, and it was not standard practice.