What retraction does and fails to do to the record
Retraction marks a paper as unreliable, but does not necessarily stop its findings from being cited or influencing later research.
What retraction counts measure
How rare retraction is and where it clusters
An analysis of PubMed retractions from 2000 to January 2010 tested this hypothesis: if misconduct severe enough to end in retraction reflects ignorance, error or naivety rather than deliberate fraud, retractions should be randomly distributed in the literature, whereas deliberate misconduct should involve a few dishonest authors or a few inadequately edited journals.
The analysis covered every English-language article with an abstract that PubMed listed as retracted between 2000 and the search date, 788 in all, found in a search on 22 January 2010.
In that sample, retraction was rare: of the 4.8 million articles archived over the decade, 788 were retracted, 0.016% or one for every 6,109 articles published. Rates by category ran from 0.009% for case reports (22 retracted) to 0.017% for clinical trials (44 retracted) and for the combined basic-and-clinical group (634 retracted), with reviews at 0.016% (88 retracted). PubMed does not distinguish basic from clinical science, so whether basic or clinical articles were more prone to retraction could not be determined; the combined basic-and-clinical group was estimated to be retracted somewhat more often than clinical trials, reviews or case reports, though still only once for every 5,895 articles.
Using 2008 journal impact factors, the 436 journals that had published an article later retracted averaged an impact factor of 6.65 against 3.12 for a comparator of 190 medical journals, a significant difference; journals retracting basic-science articles averaged higher still at 8.16, and journals retracting case reports lowest. Of all the retracted articles, 226 had first appeared in a journal with a low impact factor (2.0 or less), 378 in one with a moderate impact factor (above 2.0 and no more than 7.0) and 184 in one with a high impact factor (above 7.0). Of the 6,620 journals in the citation reports, 6,184 had no retraction at all, 376 had one or two and 9 had ten or more; on the assumption that every retracted article was equally likely to appear in any journal, a Poisson distribution predicted that no journal would reach seven, yet 10 journals accounted for 19% (152 of 788) of retractions in the sample. Several journals with more than one retraction had an impact factor below 3, which suggests that marginal manuscripts are published in journals with a relatively low impact factor, while some had impact factors above 28, which suggests that such journals may be prime targets for repeat-offender authors.
Of the first authors with a retracted article, 73% had exactly one, but seven were first author of five or more, which together made up nearly 9% (69 of 788) of all retracted articles; and where a Poisson distribution over a million equally likely first authors predicted that no author would have ten or more, two of them accounted between them for 4% (32 of 788) of all retractions. Most retracted articles had several authors, a mean of 5.1 (standard deviation 3.3); 57 (7.2%) had a single author and 105 (13%) had two. The analysis interpreted the nonrandom distribution as suggesting deliberate, potentially preventable misconduct by some authors. The analysis did not prove beyond reasonable doubt that the distribution among authors was nonrandom, though it appears to be, since the same distribution would result from a few deliberately fraudulent authors among a larger number whose inadvertent errors led to retraction.
What causes retraction
A May 2012 search of PubMed for every English-language biomedical and life science research article indexed as retracted found 2,047 of them, the earliest published in 1973 and retracted in 1977, and classified the retractions by cause. In this sample, two-thirds of retractions, 67.4%, were due to misconduct. Fraud or suspected fraud accounted for 43.4%, duplicate publication for 14.2% and plagiarism for 9.8%, while error accounted for only 21.3% and other or unknown causes accounted for the remainder, in contrast to earlier research that had suggested error was the more common cause. The analysis attributed earlier underestimation of fraud to incomplete, uninformative or misleading retraction announcements.
The proportion of published articles retracted because of fraud or suspected fraud had risen nearly tenfold since 1975. The cause varied by country of origin, with studies from the United States, Germany, Japan and China accounting for three-quarters of retractions for fraud or suspected fraud. Journal impact factor showed a highly significant correlation with the number of retractions for fraud, suspected fraud and error, across 889 articles in 324 journals (R² = 0.087, P < 0.0001), an association found in earlier research too.
A single journal's record gives a different split: by Science's own account in 2012, about a third of its retractions had been for misconduct and the rest for seemingly honest error.
Common reasons for retraction include duplicate publication of data (self-plagiarism), falsification, publisher error, authorship issues, legal reasons and results that cannot be reproduced. A December 2014 article in Science claiming that short conversations with gay people could increase support for same-sex marriage was retracted; one of its authors had allegedly faked the data. In the 2015 account, the record holder, the anaesthesiologist Yoshitaka Fujii, had had 183 articles retracted. In the 2015 assessment of Retraction Watch’s co-founder, plagiarism is not the worst offence; the more serious problem is incorrect data remaining in the literature.
Why the count rises
Science's deputy editor reported in 2012 that in the previous ten years the number of published papers retracted from scientific journals had jumped more than fifteen-fold. By the same account, Science published its first retraction in 1963, retracted eight papers in the 1990s and fifty between 2000 and 2010, among them eight by the physicist Jan Hendrik Schön and two by the stem-cell researcher Woo Suk Hwang. The analysis of 2000–2010 PubMed retractions counts Schön differently: seventeen retracted articles written in two years, nine of them in Science and seven in Nature.
Figures cited at EMWA's November 2023 conference put retractions from scientific journals at 40 in 2000 and more than 5,000 in 2022. EMWA's president at the time read the rise as both good and bad news: it suggests much more efficient control over the quality of published data, but it is also a warning to read scientific publications cautiously and critically, and not only with regard to predatory journals.
Among the 788 retractions identified in January 2010, the mean interval from publication to retraction was 23.83 months (standard deviation 21.48), shortest for case reports and longest for clinical trial reports, with a maximum approaching ten years. Both the number of retractions and the lag grew strikingly across the decade: four retractions in 2000 with a longest lag of eight months, 49 in 2004 with a longest lag of 50 months, and 184 in 2009 with a longest lag of 117 months. The analysis attributed the rise in retraction numbers at least partly to the longer lag. More of the retracted articles had originally been published in 2006 than in any other year; the analysis suggested that the longer lag left open the possibility of further retractions of papers published after 2006. Retraction Watch's co-founder gave the average time between publication and retraction in 2015 as three years, with a record on the Retraction Watch leaderboard of 27 years.
The author of the 2000–2010 analysis judged it unlikely that a cultural change in the past decade had made authors more dishonest and prompted the rise. Instead, in that author's view, editors may have become more aware of misconduct after the publicity around Schön and set a lower threshold for retraction when an article comes under question, and journals making a more aggressive effort to weed out questionable articles, even ones published long ago, may also explain why the lag has lengthened. A 2014 account attributed a big recent rise in retraction rates to journals acting increasingly quickly to remove articles found to have broken ethics rules. Retraction Watch's co-founder put the rise between 2001 and 2014 at tenfold, largely because detection was improving through detection software and "more eyes upon the problem", and counted it as some good news.
The incomplete correction record
A study reported in 2014 analysed the errata published in twenty leading journals, ten in general medicine and ten in cardiology, over eighteen months. It found 557 published errata, nearly 40% of them in the New England Journal of Medicine or The Lancet; the erratum rate correlated positively with journal impact factor; and only half of the errors classified as major had been corrected, even though 540 (94%) of the articles requiring errata had been published by signatories to the International Committee of Medical Journal Editors (ICMJE) recommendations.
A 2014 critique treated the existing recommendations as a consensus and attributed the problem instead to the inevitable failure of authors, reviewers and editors to spot every error, and the avoidable failure of journals to adhere to the existing guidance on corrections.
Of the 788 retractions identified in January 2010, 31.8% were not noted as retracted in any way by the journal that had published them. An earlier examination of fraudulent articles spanning more than twenty years concluded that efforts to correct the medical literature are often ineffective, having found retractions for only 18 of the 60 articles judged fraudulent or questionable. Retraction Watch's co-founder reported in 2015 that publishers do not alert readers that a paper is retracted one-third of the time, and that some publishers seek to hide the fact and number of their retractions because of challenges to their credibility. Where a notice is published, it can be misleading, fail to give the reasons or correct the facts, say only that the article has been retracted, or use euphemisms such as "a significant originality issue".
The harm that outlasts a retraction
Retraction does not necessarily end the damage a fraudulent article may cause. Retraction of research articles ruins careers, dents confidence in the scientific literature, and can have a profound impact on meta-analyses. Retracted papers continue to be cited long after retraction, potentially creating a false impression of the reproducibility or impact of a finding and biasing meta-analyses that include the data, and so they may influence the direction of later research; this is one reason retractions for author misconduct matter.
As reported in 2015, most retracted articles remained available and were regularly cited despite being discredited, and fewer than 8% of citations acknowledged the retraction. In the 2015 assessment of Retraction Watch’s co-founder, fraud and error in "garden-variety" publications matter much more than the major cases that attract attention, because the erroneous information is cited and repeated, and reproducibility is itself a huge issue, with recent reports that a large number of studies cannot be replicated.
The 1998 Lancet case series suggesting a link between the measles, mumps and rubella vaccine and developmental disorders prompted distrust of childhood vaccines that persisted when the paper was retracted in 2010. A subsequent epidemiological analysis in The Lancet repudiated the causal link within a year of publication, and all but three coauthors later retracted the interpretation that the vaccine causes autism. The journal retracted the article in February 2010, citing ethical lapses. The retraction specified that the claims that children were consecutively referred and investigations had local ethics committee approval had been proved false.
Many in the scientific community felt the retraction would never repair the damage. A paediatric infectious-disease chief argued in 2010 that the retraction shows how hard it is to unring the bell and to reassure people once they have been scared, and that a too-little-too-late retraction would do nothing to restore the lives of children lost in the episode. The journal’s editor argued in 2010 that the damage the paper caused reaches beyond the vaccine and autism debate to the foundation of medical communication.
Papers that are never retracted
Of 88 articles by the anaesthesiologist Boldt recommended for retraction because institutional review board approval could not be verified, nine had not been retracted two years later, according to a 2014 analysis. A critique questioned whether inability to verify approval established that the articles reported unethical research, the ground on which COPE recommended considering retraction.
When the study's authors contacted the publishers, they received a partial explanation: six articles had not been retracted because of legal threats from Boldt's co-authors. Smaller journals often lack the resources to engage in costly legal battles, which can leave editors feeling unable to retract condemned articles. Efforts to crack down on unethical practices are complicated by the threat of legal action against medical journals or coauthors who confront colleagues about questionable data.
In a case reported in January 2015, a university found convincing evidence of plagiarism in four articles by one of its former researchers, two each in two hydrology journals, and asked both journals to retract them; both declined and published editorial notes explaining why.
The executive editor of Hydrogeology Journal justified the decision in three ways.
- He played down copying substantial amounts of text without quotation marks as only “strictly” plagiarism, while acknowledging that it was unacceptable.
- He rejected adding the missing quotation marks because nobody would believe that peer reviewers had accepted papers composed largely of quoted material.
- He argued against retraction because the articles were highly cited and therefore valuable to researchers in the field.
A critique of the decision argued that it allowed extensively plagiarised papers that should never have passed peer review to remain uncorrected because they had contributed to the journal's visibility.