Are AI writing tools regulated medical devices?
AI tools that help produce regulatory documents raise two questions: whether the software itself counts as AI as a medical device or falls under the EU AI Act, and what controls the documents written with it need.
Whether a writing tool is a medical device
In May 2024 a session chair who had not yet read it thoroughly took guidance on AI that the UK's MHRA had published the month before to term AI tools AI as a medical device. The chair asked whether AI tools used in producing regulatory documents would therefore be considered AI as a medical device, and so be covered by the AI Act.
In May 2024 a device-industry data lead held that software comes into consideration for regulatory purposes only if it is software as a medical device with an AI component of one of three kinds: embedded software within a physical device, a safety component, which serves safety purposes and need not do the device's job, or standalone software. She could not then imagine how software that helps with regulatory documentation fits any of these categories, though someone might come tomorrow and say otherwise.
Whether a writing tool falls under the AI Act
For a medical writer using AI tools, the AI Act raises two questions: whether the software falls under the Act, and whether the documents written with it do.
In May 2024 one AI Act specialist read the Act this way: a provider that places on the market a system to help produce regulatory documents might be in scope, but he did not think the model itself was, because the Act always measures the system. He did not think such models would be regulated as high risk, since they do not have to comply with any of the regulations in Annex I of the Act, though he was not sure.
The device-industry data lead held that whether such software is in scope of the AI Act depends, since scope is tightly linked to the intended purpose and to the role that purpose plays in the risk-benefit assessment. She could not then name a scenario that would bring it into scope, but she could not rule one out.
What the answer depends on
Asked in May 2024 whether such tools would be covered by the AI Act, one panellist held that it was best to see how it goes, since the question requires a lot of context. Another did not know the direction, though he was sure that what happens elsewhere would influence it.
A chair of the session who was neither a tech person nor a regulator held that consideration would surely have to be given to how far along the track the AI tool is used. He set a tool that produces a first draft which humans then take up, with humans in the loop, against one that leaves the humans more out of the loop, and held that the second would surely need stricter regulation and more knowledge of the AI involved.
Controls on documents written with AI tools
In 2023 machine-learning tools were frequently used in authoring regulatory documents, particularly to reuse information from previously approved documents, such as trial protocols, statistical analysis plans, data tables and sections of final clinical study reports, in new study reports or summary documents. Used appropriately, these tools bring considerable efficiency. Used judiciously, AI and machine-learning tools are helpful and can expedite the submission process.
Such use requires vigilance from the medical writer and others in the development team to ensure the accuracy and validity of all source materials used, including data collection tools, curation and review. Listing the details of every step and including them in the electronic Common Technical Document is encouraged, so that they can be verified during review.
Effective use of documents authored with AI or machine-learning applications requires parallel review by specialists, and neither replaces the other.