A correspondence to The Lancet described how over a three-year period, 4,046 references in 2,810 published scientific journal articles had been completely fabricated, presumably hallucinations by AI. (Photo: Getty)
getty
The use of artificial intelligence or AI by researchers to write their scientific papers is far from A-OK right now. Just look at what a research team recently described a in a correspondence to The Lancet —over a three-year period, 4,046 references in 2,810 published scientific journal articles had been completely fabricated. Most of these fabrications were presumably hallucinations by AI, which kind of makes you wonder what else in those papers may have been artificial as well.
To conduct the review of what might have been generated by AI, the research team from Columbia University (Maxim Topaza, Nir Roguinb, Pallavi Guptab and Zhihong Zhanga) and the University of Eastern Finland (Laura-Maria Peltonen) turned to, well, AI. They developed an automated reference verification system. That’s because reviewing all 2,471,758 papers with 125,615,773 accompanying references from January 1, 2023, through February 18, 2026, that are in PubMed Central’s Open Access would have taken quite a bit of time without some computational help.
Comparing citations listed in the papers with actual bibliographic records helped identify and flag any discrepancies. The research team used even more AI—a large language model known as Claude 3.5 Haiku from Anthropic—to help go through everything that was flagged and separate the honest errors from the pure fabrications. Any reference that couldn’t be found databases like PubMed, Crossref, OpenAlex and Google Scholar was considered not fab and instead fabricated.
During the first year searched—2023—approximately one in 2828 papers had at least one fabricated reference. In just two years—in 2025— this had already jumped up to one in 458. Yikes. Then the first seven weeks of 2026, an even higher one in 277 paper ratio. That’s an over 12-fold increase in a relatively short amount of time.
One of the papers that deserved a particularly big F—for fabrication—was published in 2025 in an open access oncology journal and covered ureteroileal anastomotic techniques. A whopping 60 percent (18 of the 30) of the references cited in the paper were fabricated. There were other examples of papers with significant proportions of references fabricated. A total of 246 different papers contained three or more fabricated references.
The research team also identified particular authors who cranked out multiple papers with fabricated references. For example, a pair of authors had 11 different papers in a single surgical journal in 2025 that had 15 fabricated references. The research team called out so-called “paper mills,” in general. These are research groups that churn out papers The highest fabrication rates tended to be in review articles, 57 percent higher than for other types of papers (16·7 per 10 000 versus 10·6 per 10 000).
Now without talking specifically to the authors, it’s difficult to tell why these fabricated citations made it into their paper. Presumably, the authors didn’t do it for fun, as in let’s make up a scientific paper with Taylor Swift and Justin Bieber as co-authors. The suspicion is that most of the fabrications resulted from hallucinations from a specific type of AI: large language models or LLMs for short in case you don’t have time to say “arge anguage odels.”
That’s because the rise in fabricated citations did correspond with greater use of generative AI platforms like ChatGPT, Claude and Perplexity that use LLMs. I’ve covered such hallucinations in “A Funny Bone to Pick” for Psychology Today. LLMs can be particularly prone to hallucinations because they just see what happens to be associated with what in large amount of data as can be found on the Internet without really critically determining in a expert way what is accurate or not.
The problem of fabricated citations is not going to go away. In fact, it will likely get worse as more and more researchers rely AI to write their papers and more and more junk for-profit scientific journals continue to emerge. The burgeoning number of scientific journals that aim to make money but somehow expect scientists to review papers for free has led to a growing quality-control problem. Many well-established scientists don’t want to review papers or serve on editorial boards for these journals anymore, because they are frankly thankless and usually completely salary-less jobs.
One potential way of combating such AI fabrications is with AI. That means developing AI tools that can suss out inaccuracies, especially AI-generated materials. Yep, scientific publication could turn into some kind of battle of the machines. But it’s unclear if and when scientific publishers will be willing to invest into such AI tools. And how accurate such AI tools will end up being.
Nonetheless, scientific publishing is likely on the verge of a reckoning. Many publishers have seen scientific journals as cash cows, leading to the rapid multiplication of journals that are now charging researchers thousands of dollars for the “privilege” of getting their articles published. Meanwhile, funding for scientific research has been getting cut like cheese as I have described in Forbes. That leaves researchers with less and less funds to pay these publication fees. It also leaves researchers with less and less time to keep donating to scientific journals.
All of that is bound to push more researchers to cut corners and use less me, myself and more AI to do their work. That’s even before these AI tools have been fully vetted and checked for accuracy. To quote the title of that 2003 rom-com something’s gotta give.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。