Detecting and disclosing AI-generated text
A finished text raises two questions: whether a tool or a reader can tell that AI wrote it, and what an author tells clients, regulators and the readers of public-facing documents about the AI assistance used. For web content there is a third: whether a search engine treats AI-generated text differently.
Whether AI detectors work
Practitioners speaking in December 2023 and March 2024, and a speaker for The Lancet in 2024, judged AI detectors too unreliable to show whether a text was machine-written. In the December 2023 practitioner's account, a detector works on the same principle as the tools that generate text, and it makes mistakes as they do because it rests on the same models. The same practitioner said that publications had found detectors cannot detect AI text very reliably, and that a detector can help a little, but asked who would put a document through one all the time.
A false positive can get a writer into trouble for text the writer wrote: by March 2024 one practitioner knew of a couple of people it had happened to, and held that detecting AI-generated text was then not really much of a possibility. By 2024 an experienced medical writer had posted on LinkedIn that she took a writing test without using AI, that the test used another AI tool meant to detect AI use, and that she was, apparently, falsely accused of using AI. A JAMA editor had by then heard of other people falsely accused, and counted a tool that inaccurately says AI tools were used as part of the ugly side of AI use, the side everyone wants to avoid.
In 2024 The Lancet used no tool to detect generated text, because the tools were not accurate, and relied on authors to say whether they had used generative AI. In its speaker's experience, detectors often try to pick up a lack of specifics, judging that an expert would have said something different or given more detail, and that is unfair to genuine pieces on broad global-health topics of economics and policy, which can sometimes read as formulaic. Where the journal is suspicious all the same, the speaker thought, wording the question to the author and handling the exchange become harder.
By September 2023 plagiarism-detection software was being combined with machine learning to identify and quantify AI writing within a text, in versions mostly available to lecturers. OpenAI was then working on a watermarking scheme to make it harder to take GPT output without mentioning it. A journal article of September 2023 held both that AI-based writing detectors would be an extraordinary tool to uphold the writing integrity of medical articles and that AI technologies, growing more sophisticated, would ultimately be undetectable to them. The same article expected that, in the future, watermarking schemes and a legal requirement not to take any output from machine-learning rephrasing engines without mentioning it might be the answer to the problems such engines bring to academic medical writing, though this would be technically hard to apply.
Whether a reader can tell
A healthcare communications consultant held in 2023 that, as more health-related content is written to some degree with AI, consumers and healthcare professionals will find it harder to know whether a human or an AI wrote what they are reading, and that identifying AI-generated text matters because of the potential for inaccuracies or incomplete data in it.
In a survey run in March 2022, readers told AI-written from human-written health text correctly slightly more than half the time. The 164 respondents, blinded to the source, each read ten health texts: four written by authors with a healthcare designation such as MD or pharmacist, taken from websites published before the release of GPT-3 in June 2020, and six generated by AI writing assistants, such as Rytr, WordHero, Nichess, ContentBot, Texta.ai and Creator.ai. They identified the writer correctly 54% of the time, for human-written and AI-written texts alike (358 of 656 and 530 of 984). Healthcare professionals (59) and the other respondents (105) did not differ significantly on either kind of text. Medical writers (58) identified the AI-written texts significantly more often than the other respondents (106) and did not differ from them on the human-written ones. Correct identification of single texts ranged from 30.5% to 82.3%. Two guest editors of the journal issue that published the survey took one such survey: one identified four of the six AI-written texts and the other none. In a LinkedIn poll of 66 people, 59% were confident that they could tell the difference, several of them commenting that it would be highly evident to them, 29% did not believe they could and 12% were uncertain.
A journal speaker asked in 2024 whether AI use is always obvious held that it is not, and that how obvious generated text is to a reader depends on how the tool was used. Output that a non-expert copies and pastes from a prompt is, in that speaker's view, normally pretty obvious: it is loose and unspecific, its language is hyperbolic, and it has no nuance and no insight. Where the model served as a co-creating partner, giving ideas that the writer then wrote around, its use is very difficult to tell at all.
Editors at Taylor & Francis listed in 2024 what stands out to them in a manuscript as suspected AI-generated text; a sign does not always mean AI was used, but a lot of times it may, and when the editors ask the authors it comes out that AI was used. The signs are:
- tortured phrases, complex words and non-scientific language, such as "orchestrated by malignant forces"
- repetitive writing
- surface-level discussion of scientific issues, in a review with no real critique or opinion in it
- a generic conclusion that refers to none of the points made earlier in the paper
- a style different from the same author's emails and responses to reviewers, which can also mean two authors with different styles
- paragraphs written with lists that start with a semicolon
- an abbreviation defined every time it is used
Telling clients about AI use
A medical writer speaking in May 2024, as no lawyer, advocated complete transparency with clients and held, as a fundamental belief, that clients, patients and anyone else should be informed if an AI tool is used for anything; the writer would want to know too. That writer's clients were all told and mainly saw it as a plus, so it was not a difficult conversation. Whether it needs to be in the contract depends on the contract with the client and is a legal question, on which the writer would take professional advice.
Declaring AI use to regulators
At the EMA, a speaker from the agency reported in May 2024, whether it would be required or useful to declare AI use in a particular document had been discussed with almost no conclusion, and the speaker could not even say it was moving in a direction. The speaker suggested this could be because humans remain responsible for the document either way, which leaves open whether a declaration would be relevant.
Telling the readers of public-facing documents
Whether the readers of public-facing documents such as plain language summaries and informed consent forms need to be told that AI was used drew three answers from a panel in May 2024. The first treated it as an ethical question regardless of context: the user has a right to know, the speaker understood the final text of the AI Act to require that the user be told where an AI agent is involved, and by extension documents such as device instructions for use and summaries of safety and clinical performance should say so clearly. Where AI is used to turn such a document into the lay language a patient can read, the same answer held that writers should at least move towards clearly mentioning that AI assisted.
The second answer, while not knowing exactly what the regulation would say, seemed to remember an exemption from mentioning AI where a human is in the loop, modifies the text, publishes it and is responsible for it. On that view, taking AI output as one input is no different from reading a lot of text and writing from it, provided no factual information or sources are taken from the model, and there is then not much reason to cite it as a source. It depends on how far the writer relies on the tool, though: text that is simply copied and pasted should definitely be marked as AI, which the speaker believed to be obligatory.
The third answer, from a lawyer, saw the risk in sources: text created by a language model comes without knowledge of its sources and without quotation, and that is the real problem for transparent scientific work and publication. Using AI to support one's own work as an author is not really a problem if the part contributed by the natural person prevails, and establishing that is really difficult.
In a routine half-hour review of a client's plain language summary of a publication, one translator found the text so far below that client's usual standard that the reviewer was confused, and only at the bottom of the document found an acknowledgement that it had been produced by ChatGPT. The reviewer was both horrified and delighted, applauded the client for the disclosure, asked the client not to do it again and, if they were going to use the tool, at least to have a human review its output before it came anywhere near the reviewer, and was annoyed at having worked through it all rather than looking at the bottom first.
In May 2024 a group involved with CISCRP was gathering the public's view on whether the readers of such documents should be told that AI was used. In a survey presented the same month, over 75% of respondents thought at least some pharmaceutical companies were already using AI to write patient- and public-facing documents, and 93% said it was at least somewhat important to make clear if and how AI had been used in them.
The author of the 2022 survey concluded in 2023 that its findings suggest the authors of health-related content should be required to identify whether their work was completed by a human or an AI writer. The same author held that the findings suggest it is important for organisations to establish clear guidelines on the use of AI writing assistants in healthcare, and that, as more people gain experience of AI writing tools, such guidelines on their acceptable use will become increasingly important to address the potential for misuse or abuse of the tools and the need for transparency and accountability in their use, particularly in healthcare.
Showing that a human checked the text
A panellist in May 2024 asked how a first draft generated by AI and then 99% changed could be disclosed other than by saying exactly that, and held that the best way to disclose AI use on documents had still to be worked out, so that people trust not only that AI generated some of the text but that a human checked it, which is the most important part. Another panellist pointed to a template that already exists: the ICMJE authorship criteria and the checklist on which each individual indicates their role and responsibility, and thereby whether they qualify as an author. The same panellist saw no reason it could not be recycled for AI contribution and let the reader beware: a full disclosure of what was done and how, with the reader's trust resting on their own confidence in the system.
Where disclosure is heading
One practitioner held in December 2023 that declaring AI use is like declaring conflicts of interest: there are guidelines, forms, committees and societies, yet not everyone declares, it has no legal consequences, and the practitioner had never or almost never heard of a paper retracted for an undeclared interest. The same practitioner likened AI tools to a calculator, which nobody declares in every school test, and thought that was the direction things would go. In that practitioner's personal opinion, which the future might prove wrong, it will in the end become standard to use AI and to declare that it was used and that everything was checked, which a medical writer would do anyway, and the present fuss over the tools is a big one for something that will become very normal.
A medical writer said in May 2024 that it would be no surprise if the recommendations of an ethical paper on AI being prepared by an international medical organisation included a requirement to disclose fully the use and the degree of use of AI in generating medical and scientific information. The same writer expected the ICMJE to follow and the European and US regulators to require it too, because their reviewers, looking at the data point by point and in aggregate, will want to know how much scrutiny to apply to fields an AI tool may have influenced; slow as such bureaucracies are, they would get there in very short order. MHRA guidance had been published that April. EU guidelines on generative AI use had just been announced, and another panellist expected the EMA to come next.
How search engines treat AI-generated text
In June 2025, according to a writer of health blogs for clients, Google did not try to detect whether web content was generated by AI and did not penalise it as such, and using AI as a tool for web content was not wrong in itself.