Generating images with AI
An AI image generator turns a written prompt into a picture, which puts slides, infographics, posters, illustrations and graphical abstracts within reach of a writer without the skills to make them by hand. In 2024 the generators produced general and promotional images that practitioners used, struggled with graphical abstracts, and could not yet produce accurate scientific illustrations.
How a generator turns a prompt into an image
Using AI, machine learning and neural networks, a generator converts textual descriptions, or prompts, into digital images, which makes visual elements more accessible and faster to create. The generator is trained on an extensive amount of text that a natural language processing model translates into numerical representations, and those representations are guidelines that help it interpret a text prompt and represent it visually. The generator does not "understand" the content of the text, but predicts content from patterns observed in its training data.
What writers use generators for
Creating visual assets requires specialised knowledge and skills, and the time to acquire them, and many medical writers may feel daunted by the specialised skills that high-quality visuals require. AI can support novices and even experts in developing eye-catching visuals that hit their mark.
AI is used to generate images in an increasing number of fields, science communication among them. The New York Times has used AI-generated content for visuals tracking the spread of COVID-19, and The Washington Post for informative maps and charts. NASA's Mars rover uses AI algorithms to analyse the rover's data and create 3D models and topographical maps of the surface of Mars. Advertising campaigns for brands such as Nutella, BMW and Nike were created with AI with great success. In finance and education, AI applications have been created for data visualisation and for educational material. IBM's Watson Analytics uses AI algorithms to create charts, maps and graphs for analysing trends, and Google's TensorFlow Data is a group of tools for producing histograms, scatter plots and heat maps.
In science communication and medical writing, AI tools can speed the creation of slide decks, infographics, conference posters, illustrations for articles or for teaching, graphical abstracts, diagrams, and images illustrating methods and concepts in scientific articles. Several online services already provide AI-powered tools for creating visuals to academic institutions and biotech companies. Concerns about how these constantly evolving tools are used to create scientific content have prompted discussion of how accurately AI represents scientific notions. The intellectual property ownership that must be attributed to the original artists and illustrators is also debated. Microsoft 365 Copilot is an evolving tool that could help overcome some of these concerns.
A freelance medical writer and the cybersecurity specialist who runs the cybersecurity operations of the medical communications company she writes for and leads Copilot's deployment in clinical research and medical writing environments had used Copilot's graphic generation and graphic recognition for four uses. Asked to write legends for diagrams and illustrations, Copilot was so good at identifying what was going on in an illustration and describing it relevantly that they can use its descriptions as legends. As freelancers they run frequent promotional campaigns, and since they have had Copilot it has created all of their promotional illustrations. For each client engagement they create a dedicated, secure team in Microsoft Teams and a special logo for the project, which they hold important for their branding, and virtually all of those logos were by then generated by Copilot. One, for a research paper in regenerative medicine in the first quarter of 2024, showed DNA and laboratory equipment, its light blue above and darker blue below suggesting the research's innovative before-and-after flow, and was made with the prompt "Create a logo emphasising the [core innovation]". They had created many presentations with Copilot in PowerPoint and were quite impressed: it made an advertisement for presenting a poster at a conference entirely by itself, and produced the structure, slides and pictures of a manuscript summary.
Another freelance medical writer uses Copilot for inspiration for the images and illustrations she needs for her YouTube channel and books. Copilot can create basic PowerPoint presentations and infographics, but she creates no scientific illustrations with any AI tool, judging them not yet advanced enough.
A freelance scientific graphic consultant with eleven years' experience, who trains others in graphics, uses Copilot for images for LinkedIn posts, infographics and PowerPoint presentations, and for introductory and general images, such as when introducing a scientific topic to students, but finds that it cannot create specific scientific visualisations.
Copilot and its Designer tool
Copilot's tool Designer creates images from prompts using DALL-E, the text-to-image generator developed by OpenAI, which converts text descriptions generated by Copilot into images. DALL-E 3, the latest version in 2024, released in October 2023, increased the number of image generation boosts from 15 to 100 a day and aims at more accurate results. Copilot is straightforward to use and can create very detailed, high-resolution images in a variety of styles, and by 2024 it had advanced rapidly since its launch to become one of the best AI image generators available online. In Whiteboard, Microsoft 365's collaborative canvas for brainstorming, Copilot's content can be used to create images, and in Excel Copilot can make illustrations from tables of data.
For the writer and the cybersecurity specialist, the most dominant advantage of Copilot for Microsoft 365 over other AI tools is that it understands a user's business context. It accesses the user's emails, OneDrive files and SharePoint pages and uses them in preparing its responses, so it can produce images far more relevant to the user than a tool without such access.
Copilot's top limitations include the time to be invested in trial and error before the desired result is reached, and its user must be aware of the risk of creating deep-fake images. Updates announced in April 2024 for Copilot Pro and Copilot for Microsoft 365 licences were to increase the number of images a user can generate from 15 to 100 a day.
Choosing a generator
Among the most used AI image generators in 2024 were DALL-E, Midjourney, Stable Diffusion and Canva. All were evolving rapidly, and each had distinctive strengths that could meet users' needs differently, so the comparison below does not rank them.
| Generator | Model | Features | Free trial |
|---|---|---|---|
| DALL-E (Copilot) | Integrated with ChatGPT | Easy to use, ideal for beginners; advanced ability to convert text to images; high variety of image styles; high-resolution, realistic and detailed images; fast image production | Yes |
| Midjourney | Proprietary machine learning model | Abstract and artistic style; extensive customisation; high range of resolutions and editing tools; intuitive user experience; user-friendly interface; connected to an active community of users | Yes |
| Stable Diffusion | Open source | Extensive customisation; fast and efficient in creating high-quality images; consistent and reliable results; user-friendly interface; most suitable for users with technical knowledge; less creative | Yes |
| Canva | Web-based design service | Simple and intuitive; rich set of features; limitations in replicating complex designs; limit of 100 AI-generated images a day; flexibility in editing images | Yes |
The writer who makes images for her YouTube channel and books finds that Copilot can help brainstorm ideas for images as a starting point, but that for more sophisticated images the time spent tweaking the prompt is not worth it. Because prompts take time before they give what she wants, for now she prefers other AI tools dedicated to image generation. When something has to be created from scratch she uses Midjourney, though she first tries to find good images in Canva and Freepik. Midjourney is the AI tool she uses most to create her content: it is fast, and at the moment she prefers it to Copilot.
From a technical point of view the graphic consultant's favourite generator is Midjourney. With a properly detailed prompt and the right settings it gives her the desired image and generates very detailed images, but unlike Copilot it does not prevent the creation of controversial images. She describes two features of DALL-E. Its settings can be customised for highly tailored images, taking instructions that are added automatically to every prompt, so that a series of images can keep one objective style without repeating the same basic commands each time. And, as a key difference from the other AI tools, a user can dialogue directly with ChatGPT, without entering the image generator program, so that ChatGPT writes the prompt, which can be much richer and more detailed than the user's own, and submits it to DALL-E. DALL-E nonetheless needs very detailed, clear commands to create the desired image.
Graphical abstracts
Graphical abstracts, the fourth use the writer and the cybersecurity specialist made of Copilot's graphics, were in their view still a work in progress: they worked hard on different prompts, but the results were mixed at best. They think DALL-E is stubborn. Given an abstract and asked for a graphical illustration, it came up with a surprisingly sophisticated image and then stuck to that initial concept however the prompt was adjusted, once producing an image that looked like an infographic when the format of a graphical abstract was wanted. They suggest two possible reasons. The first is that grounding cannot be used with DALL-E through Copilot. Grounding is the process of using large language models with information that is use-case specific, relevant and not part of their trained knowledge, and it is crucial for the quality, accuracy and relevance of the output. When they offered a graphical abstract template as a reference file for grounding, DALL-E would not accept it. The second is that among the millions of internet pictures used to train DALL-E there are far more infographics than graphical abstracts, and an image search turns up few good graphical abstracts, so they think it has not been trained specifically to generate graphical abstracts.
They are sure Copilot will create graphical abstracts eventually. It already produces infographics, which they are sure will keep getting better, and they think it will be able to create molecules, cells, proteins, test X-rays for education, body organs with a focus on a mechanism of action, and illustrated guides to administering a treatment, with, in their words, no limit.
Scientific accuracy
In the graphic consultant's view, Copilot, like the other AI image generators, cannot be used for creating precise and accurate scientific content for peer-reviewed journals or in other contexts where scientific accuracy is fundamental. It can provide context or a reference point, but it cannot yet be used for data visualisation.
She set images made with Copilot's DALL-E against images used for human health fundraising web pages and against drawings by a scientific illustrator. Set against the fundraising images, DALL-E made the introductory, general kind of image she says it helps her generate. Its prompts were for two hands holding a clod of earth from which two kidney-shaped seedlings grow, against a backlit sky, in a balanced or a precise style, and for an image for a fundraising project for the study of a new drug to cure kidney disease.
Asked for a "human kidney section", DALL-E produced an illustration with multiple mistakes, set against a real microscopic image of a human kidney section and an illustrator's drawing:
- The external tissue around the kidney is adipose and should be more regular, smaller and homogeneous.
- The ureter is not interrupted.
- The renal pelvis should be empty.
- The medullary and cortical stroma are characterised by ray structures of nephron ducts and tubules.
Asked for a "histological renal section with glomerulus rounded by podocytes", it produced an illustration with multiple mistakes, set against a real image of a renal glomerulus section and an illustrator's drawing:
- The tissue outside Bowman's capsule is characterised by epithelial tubules.
- The structure inside the glomerulus is one single capillary ball, not several epithelial bean shapes.
- Bowman's capsule must be empty.
- The glomerulus has two exit points, the hilum for the blood system and the proximal tubule for urine collection.
- The proximal tubule does not stop but continues to the collecting duct.
Tracing the sources of the data and images used by AI tools was not yet possible in 2024, and generating falsified, manipulated or scientifically incorrect data is considered a major threat to scientific integrity, especially among those in the scientific community. Caution is still needed about when the tools are applicable to use. Their rapid advance, particularly in quality and data protection, raises the question of when similar progress will be seen in appropriate copyright regulation, privacy, and a reduced risk of inaccurate scientific fake images. As Copilot and other AI tools evolve so quickly, innovative developments are easy to imagine that could revolutionise how visual content is created.
The graphic consultant is impressed by how rapidly the tools are evolving and thinks they have great potential. She suggests that over the next few years AI may create more accurate scientific content. In her view, that would raise debates on the ethics of artificially generated images in specialised scientific communication, and make the longstanding issues with the peer review system more urgent to address, especially as open-access publishing grows.