How health coverage goes wrong and how it is judged
Health coverage goes wrong in recognisable ways: words that promise more than the evidence gives, stories that convey a certainty the evidence does not have, and a skew in what a field's coverage reports and whose research it cites. A health story can be graded against ten published criteria, from whether it quantifies the benefits and harms of an intervention to whether it relies on a news release, and a team of reviewers rated most of the stories it graded unsatisfactory on half of them. Journalists, in one critic's view, should help the public understand the trade-offs that health care decisions involve.
The words that overclaim
The health-journalism scholar Gary Schwitzer names seven words that reporters should never use in medical news: miracle, cure, breakthrough, promising, hope, victims and dramatic. In research in 2002 he found almost a thousand stories about experiments to develop a drug for the common cold. A third of them called it a miracle, a wonder drug, a super drug or some other sensational term, and many predicted imminent FDA approval. An FDA advisory committee then rejected the drug unanimously, and few if any of the original reports followed up on the rejection prominently.
According to one media critic, in 2013 NBC News reported a cure for baldness on the strength of a mouse experiment with five successful results, and the same year the New York Times, Time and Esquire all alluded to a cure for cancer. In 2012 CNN proclaimed a cure for cancer within reach on the basis not of new findings but of a new programme at a medical center. Although only about 8% of drugs in phase 1 trials will be approved for marketing, health news stories often describe phase 1 results as breakthroughs, including in network television reports on ABC, CBS, CNN and NBC. A Prevention magazine cover proclaimed a "cancer vaccine breakthrough" over a sixteen-word story about a drug just moving into testing. Such coverage harms people by misleading them and giving them false hope.
How a health story is graded
Who grades health stories
In 2012 HealthNewsReview.org's publisher led a team of experts who graded daily health news reporting by major US newspapers, wire services and the websites of major news outlets. In 2014 the team numbered more than two dozen people. It was established in 2006 and designed to show how coverage of health care and reporting on the claims of researchers should be scrutinised, and in 2014 it was the only health care news watchdog service of its kind in the United States. Its reviewers came from journalism, medicine, health services research and public health, and they used ten criteria to review news stories that make therapeutic claims about treatments, tests, products and procedures.
The ten criteria
The ten criteria ask whether a story:
- adequately discusses the costs of the intervention
- adequately quantifies the benefits of the treatment, test, product or procedure
- adequately explains and quantifies the harms of the intervention
- seems to grasp the quality of the evidence
- commits disease-mongering, meaning that it exaggerates or oversells a condition, or its seriousness or prevalence
- uses independent sources and identifies conflicts of interest
- compares the new approach with existing alternatives
- establishes the availability of the treatment, test, product or procedure
- establishes the true novelty of the approach
- appears to rely solely or largely on a news release
How graded stories have fared
Of 1,889 stories reviewed over seven years, from April 2006 to May 2013, the reviewers rated most unsatisfactory on half of the criteria: costs 69% unsatisfactory, benefits 66%, harms 65%, the quality of the evidence 61%, and comparison with existing alternatives 57%. The stories came from newspapers (43%), wire and news services (30%), online outlets including those of broadcast and magazine companies (15%) and network television (12%).
How a story conveys a certainty that does not exist
A "daily drumbeat of dreck" is, in Schwitzer's words, the mainstay of much health care journalism. Instead of helping readers and viewers understand the evidence behind treatments, tests, products and procedures, many health news stories emphasise or exaggerate potential benefits, minimise or ignore potential harms, and ignore cost. Many health journalists, he holds, write as if all the evidence is in, especially for newer and costlier interventions, when it almost always is not. Health care journalism has unprecedented peaks of excellence, in his view, but the valleys between them are becoming wider and deeper and may overwhelm the good done at the peaks.
Conveying a certainty that does not exist was the commonest flaw that HealthNewsReview.org's reviewers found. A story conveys it by exaggerating the effect size, using causal language to describe observational studies, failing to explain the limitations of surrogate markers and endpoints, relying on a single source with no independent perspective, and failing to analyse the quality of the evidence independently.
Press releases and news stories that misrepresent randomised controlled trials add to the drumbeat of dreck. A 2012 study by Yavchitz and colleagues in PLOS Medicine found positive spin, reporting that emphasised the beneficial effect as stated in the abstract's conclusion, in about half of the press releases and news stories it examined. Schwitzer stresses not believing the stated bottom line without confirming it in the data. Many journalists lack the knowledge to confirm it, and in his account do not understand terms such as composite endpoints, surrogate markers, subgroup analyses and inadequate comparators. In Schwitzer's words, "We have entrusted the reporting of health stories to the public to many people who are ill-prepared to know bad science."
The skew in what a field's coverage reports
Beyond the oncology clinical study reports and manuscripts that medical writers prepare lies a body of cancer literature aimed at a lay audience, and it may paint a completely different picture of the disease from the one in the tables, figures and listings.
Two studies examined how the media in two countries cover cancer research. The first analysed the content of cancer news in eight US newspapers and five magazines between 2005 and 2007. Of 436 randomly sampled articles, 32.1% focused on survivorship and only 7.6% on mortality, 57.1% covered aggressive cancer treatments while only 13.1% reported that such treatments can fail, and two covered end-of-life palliative care. The authors criticised the American media for misleading the public with an "inappropriately optimistic view of cancer treatment, outcomes, and prognosis".
The second analysed BBC stories on cancer research from July 1998 to June 2006. The stories' favourite topic was innovations in treatment at 20%, followed by lifestyle choices at 12%, genetics at 9% and nutrition at 8%. Most stories cited peer-reviewed articles, but with a bias toward UK papers at 40% and US papers at 36%. British papers were over-cited by a factor of about six relative to research papers from the rest of the world. Three high-impact journals dominated the sources: The Lancet, the British Journal of Cancer and the British Medical Journal. The authors concluded that the BBC's reporting of cancer research was narrow relative to global cancer research activity and output.
Both studies found breast cancer the most covered cancer type, at over 30% of all BBC stories, and over-reported relative to its disease burden of 13%. Its high survival rate, which supplies happy-ending material, may explain this preference. The survivorship of young, glamorous celebrities, such as Kylie Minogue and Christina Applegate, also glamorises the disease and keeps the press busy. An estimated one in two men and one in three women will develop cancer during their lifetime, and everybody knows at least one person diagnosed with it. Cancer is so widespread that it has almost become a household word, so balanced and less hyped reporting on it is all the more needed.
Journalists should critically assess and challenge the status quo, in the view of one writer from inside the medical device industry, and the headline-hunting and biased reporting of the "implant files" on medical devices was a missed opportunity, which worst of all missed chances to tell the public about sources of reliable information. The same writer holds that the journalists who complained that devices had reached patients too early would, always with a selling headline in mind, have reported that patients in a US device trial were randomised to a death sentence had they learned that the comparator arm fared worse. The coverage also cited physicians with concerns about industry. As a reader, the writer wants to be given facts and to form their own opinions rather than be fed the opinions of others. With online media the public has become used to reading web-based headlines, perhaps missing more reliable sources of information, and the speed of the news cycle may put journalists under increased pressure to get stories. The writer hopes for a way back to balanced reporting.
What journalists should help the public understand
Too much of the wrong kind of health news matters, in Schwitzer's view, because the public debate about health care is ugly and uninformed by evidence-based medicine. When the US Preventive Services Task Force followed the evidence and recommended against prostate-specific antigen screening in asymptomatic men, the chief executive of Zero, a nonprofit working to end prostate cancer, accused it of condemning "tens of thousands of men to die this year and every year going forward". When the vice president of the National Breast Cancer Coalition, herself a breast cancer survivor, told an FDA advisory committee that the data do not show Avastin extending the lives of breast cancer patients but do show it increasing the risk of harm, she was reviled, and one person at the meeting called her "an embarrassment to all cancer survivors".
In Schwitzer's view, journalists should help people understand three things: that in health care newer is not always better, more is not always better, and less can be more; that bad things happen when normal states of health are redefined as illnesses requiring treatment; and that every health care decision involves a trade-off between something gained and something lost.