Public trust in AI-generated health information
The public's trust in health information and patient-facing trial documents produced with AI is moderate and conditional: surveys find people willing to use such information yet worried about its accuracy, and trusting a document more the more a human was involved in it. That trust also depends on the reader's own health situation, on what they know of how the information was made, and on what the technology is attached to.
Why public trust matters
Trust is a cornerstone of efficient communication in healthcare, and its erosion results in poor patient outcomes through disengagement from healthcare services. Although AI has proven its benefits in patient education and diagnostics, public doubt about it remains. AI's capability to generate new health information raises concerns about its accuracy, its transparency and the risk of hallucinations, in which incorrect or fabricated data are generated. Meeting these concerns is essential if AI developers are to keep developing better versions of the tools for use in healthcare and decision-making.
As reported in 2024, 60% of Americans overall indicated discomfort with the use of AI in their care. Because deep learning creates its own algorithms, its models can lack transparency, and this black-box effect enhances distrust of AI in many population segments. It can help to keep this discomfort front of mind when writing for lay audiences.
Many patients now take a partnership role in decisions about their healthcare, in the view of one communicator who led a May 2024 survey on AI and clinical research documents. Such decisions were once the province of prescribing physicians, in a more parental relationship in which the patient asked the doctor what to do, and many patients now look on the web instead. With AI, he holds, humans are no longer necessarily involved in the genesis or the oversight of information posted on the web, so people access information without understanding how much, or whether, a human was involved. People making critical decisions about their healthcare are then relying on information that may or may not come from a source they have traditionally held in high confidence. Patients at sea in a world of information overload will not use information they do not trust, and the greater its integrity, the better prepared they are to evaluate the data that apply to their situation and reach a more informed decision.
AI is here to stay, and in May 2024 the survey's lead put whether to trust it as an open question. AI plays a significant role in output now viewed by prospective patients, decision makers and care providers, and in his view trust in it starts from a position of scepticism.
What trust depends on
In the survey lead's account, trust is contextual: whether a patient's condition is untreatable by the therapies currently available, whether they avoid or tolerate risk, and whether they are averse to pain or have a high pain threshold all determine the path they might take to a decision about their healthcare. In many examples, patients with very serious, life-threatening or progressive untreated conditions are more and more willing to take risks that people in less dire circumstances might not, so how desperate the patient is becomes part of the equation.
AI programs are often used without full disclosure of how they are used, and patients have no idea what information an AI tool or large language model has scoured, what algorithms and keywords guided its searches, or whether its output was generated with or without human intervention and oversight. So, on his account, using these tools for health information takes a leap of faith. The technology is evolving very rapidly, and its sophistication and invisibility are likely to grow. He finds that invisibility frightening, because he does not know how the technology is being used, how far it is used autonomously, or whether a human intervenes or oversees it.
AI having issues does not mean that human-generated information had none, he adds: it has carried biases, cherry-picked data and faked photomicrographs. Not all misinformation is machine-generated, and the huge damage caused by vaccine scepticism arising from Andrew Wakefield's studies associating the MMR vaccine with autism had nothing to do with AI. Given the human tendency to cut corners, make up information and insert biases, whether information is any worse or any better with AI, and whether AI provides some protection against these tendencies, remains to be seen.
Who the surveys asked
An online survey run in May–June 2024 recruited through Instagram and through friends and family, targeting people aged 18 or over who had accessed health information within the past three months. It was open to participants anywhere, but the social networks used for recruitment were in Kuwait and the UK. Of 75 responses, 60 were included in the analysis. Of the respondents, 65% were aged 18–24, 15% were 25–54, 11.7% were 55–64 and 8.3% were 65 or older.
The May 2024 survey was run by an organisation that encourages participation in clinical trials and had 1,010 respondents from Europe and the United States, people both with and without clinical research experience. It used a validated instrument, sent to patients in North America and Europe, to gauge where people sit on a spectrum of trust and to read that against their demographics, their socioeconomic status and their comfort with technology. It asked about trust in the pharmaceutical industry, familiarity with AI and with clinical research documents, and the use of AI to create those documents, including trust, accuracy and disclosure of AI use, with open questions on fears and benefits. The clinical research documents meant were mostly informed consent forms, clinical trial results summaries, plain language protocol synopses and other informational brochures and content.
The results presented in May 2024 were preliminary, with data still coming in. The breakdowns by age, gender, race and education level, and by knowledge of AI and exposure to different patient-facing documents, had not yet been done. The survey was not global and not representative of all of humanity, but a snapshot in time of the level of trust among a very select group, giving at least some sense of what one cohort felt about data generated by AI.
A computer-based survey of 2,052 people aged 14 and over in Germany, reported in December 2019, asked about their use and perception of technologies such as the internet of things, artificial intelligence, algorithms and bots.
Trust in AI-generated health information
In the May–June 2024 survey, the public's perception of AI mixed optimism and scepticism: 68.5% of respondents were willing to engage with AI-produced content, but there were significant concerns about the accuracy, biases and ethical implications of the information. Respondents knew what AI systems can do, but their trust in AI-generated content was moderate. Most respondents had a moderate understanding of how AI produces health information. Of the respondents, 27.8% would read AI-produced health information without hesitation, while the majority were open to its use but concerned about fully trusting it. Accuracy was a concern for 83.3%, who worried that AI systems might provide inaccurate information resulting in poor health decisions, and this concern was compounded by AI hallucinations, which many respondents did not know about. One respondent added that most people are unaware that using the right prompt extracts much more accurate healthcare information. Respondents were also concerned about privacy, in particular unauthorised access to health information, and about the transparency of AI systems' decision-making. One respondent noted that information may be biased by region and personalised to a specific area.
Most respondents searched for health information out of general curiosity, to manage their own or their family's health, or after a recommendation from a healthcare professional, which often led them to explore symptoms, treatment and preventive measures. Managing their own or their family's health stemmed from patient empowerment and self-care, and from the importance of sharing health information with others.
Younger respondents were more trusting of and familiar with AI-generated health information than older ones (t = 2.14 [58]; p = .036). Those aged 18–24 showed the highest trust and had more exposure to AI technologies such as chatbots and digital health platforms, and 75% of them were aware that AI can generate health information, though some still had concerns about its accuracy. Older respondents were less aware of AI's capabilities, and many were sceptical of its accuracy and more inclined to trust advice from healthcare professionals. The survey's author attributes the older respondents' mistrust to lower digital literacy, limited exposure to AI technologies and concerns about data privacy, and the younger respondents' higher trust to their exposure to technology, and holds that a lack of exposure can limit trust in and acceptance of AI.
Respondents named the main benefits of AI-generated health information as efficient and rapid access to information, reduced workload for healthcare professionals, personalised health advice and up-to-date information, and most were optimistic about AI's potential to improve healthcare. One respondent found AI-produced health information convenient for starting research but always double-checked it against trusted sources such as UK guidance. The factors shaping respondents' perception included a clear explanation of how the information was produced, evidence of its accuracy, endorsement by healthcare professionals and the ability to give feedback. Transparency, endorsement by healthcare professionals and clear evidence of accuracy proved critical in building trust in AI-generated health information. One respondent said that AI needs more endorsements from healthcare providers.
Trust in AI-made clinical trial documents
In the May 2024 survey, 75% of respondents reported being at least somewhat familiar with AI and 64% had used it for something, from random questions about life to business, with ChatGPT and Meta AI among the tools named. And 67% agreed or strongly agreed that AI always produces accurate and appropriate text and images. The presenters offered two possible explanations: people may overstate how familiar they are with AI, since the more one uses it the more one finds it is not always accurate; or they may be judging it by what they use it for, where a search covering a topic or two may seem accurate, while in interpreting scientific data it becomes a lot less accurate and a lot more fallible.
Of the same respondents, 95% said it was at least somewhat important for humans to be involved in reviewing or developing patient- and public-facing clinical trial documents. Asked how much they would trust a document written entirely by AI with no human review, one created by AI and reviewed by humans, or one written only by humans, respondents' trust rose with human involvement, at least for now, and nearly a quarter would not trust a document at all if only AI was involved. Trust rose with human involvement even though most respondents had agreed that AI is always accurate.
Asked what value they saw in using AI to create patient- and public-facing documents, respondents answered anything from little value to lots of value, including that it can be useful for correcting errors and giving ideas. Asked what fears they had, they answered that errors could be fatal or that the communication could be too impersonal, while some had no fears at all. These comments ran the whole spectrum on whether AI does more harm than good and how accurate it is, and the survey's lead found them very similar to those he had heard from the people attending the conference where the survey was presented. He concluded that being a member of the general public or a patient does not seem to be a discriminating factor in how intuitively one trusts machine-generated information.
Comfort with different technologies
In the German survey, about 20% considered technologies such as the internet of things, artificial intelligence, algorithms and bots rather positive, 11% rather negative and 38% felt indifferent or neutral, while 27% did not know what they meant. The younger the respondents, the more open-minded they were towards new technologies, and people with a low educational status were the least open-minded.
On data from Initiative D21 published in 2019, people felt rather uncomfortable with digital home assistance such as Alexa (44% rather uncomfortable, 24% mixed feelings, 13% rather comfortable, 19% did not know) and with robotic assistance at home, in hospital or in nursing care (57%, 27%, 11% and 5%), but rather comfortable with implants that release active substances into the body (25%, 28%, 41% and 6%). People felt rather comfortable with medical devices that deliver medication to the body, supervise clinical parameters and inform medical staff in an emergency, while acceptance of robotics seems limited. The 2019 report reads these results as showing that digitalised services meet scepticism across a broad part of the German population, while there seems to be more trust in digital health services, perhaps because of the importance of personal relationships and trust. Personal interaction is the foundation of most forms of health care, its relationships rest on information and mutual trust, and both seem to strongly influence the acceptance and use of digitalised innovation in health care.
Building trust
The human component is very important, in the survey lead's view, and knowing how and how much AI was used to generate information strongly influences how much one trusts it. The pharmaceutical and healthcare people who create the information people use need to be trusted, not just the tool, so that they are not seen as manipulating information in a way that could be perceived as harmful. In his view, the more their reliability and trustworthiness are fostered, the more trust is conveyed to the use of the tools in generating critical information. It can be life and death, because people go to chat groups on the web and turn to claims such as that hydroxychloroquine will cure their covid or that drinking some bleach will be fine.
The author of the May–June 2024 survey recommends future research specific to older people, whose trust in and familiarity with AI vary, and holds that user-friendly tools may help healthcare providers and patients understand AI-generated health information. Targeted educational campaigns, or more user-friendly AI tools designed for older people, may be necessary to bridge the trust gap between age groups, and the author holds that healthcare providers and AI developers must focus on transparency, accuracy and ethical considerations to address it. Addressing AI hallucinations could increase public trust and ensure patient safety, and educational programmes to increase patients' literacy around AI tools could help them comprehend AI's capabilities and limitations. Users' awareness of the algorithms and data types behind AI, such as medical or peer-reviewed databases, can help foster trust in and acceptance of AI technologies. Cross-referencing AI output with trusted medical outlets could help enhance its accuracy and users' trust, and incorporating feedback from healthcare providers could help refine AI applications by making users feel more engaged. Increasing public awareness is, in the author's view, crucial to limiting public scepticism of AI tools, through educational programmes on the foundations of AI tools in healthcare, collaboration with trusted media outlets to reach a broader audience, and the involvement of healthcare providers in supporting the technologies.