Adding an element of human touch could be the key to avoiding AI model collapse, new research finds. (Image credit: Getty Images) Share this article 0 Join the conversation Add us as a preferred source on Google Newsletter Subscribe to our newsletter While the evolution of artificial intelligence (AI) systems has shown no sign of slowing, there's a growing concern that large language models (LLMs) will soon run out of human-made data to ingest and learn from.
Once this happens, scientists say, AI models will increasingly rely on synthetic AI-made information, which will lead to an effect called "model collapse." This is where LLMs spout gibberish and the AI systems they underpin deliver inaccurate answers and hallucinate information to queries far more commonly than they do today.
"That's especially worrying considering some experts think that we will run out of high-quality human-generated data by the end of the year — so if you're relying on this synthetic data, but there's an almost existential threat it will sink your AI, you're in trouble," Yasser Roudi, a professor of disordered systems in the Department of Mathematics at King's College London (KCL), told Live Science. "If, for example, you had LLMs that were used in hospitals to analyze brain scans and find cancers, if while training another model they experienced model collapse, these machines could misdiagnose people."
However, Roudi recently found that model collapse can be bypassed by adding a single human-made data point to an AI's training data, even if all the other data is AI-generated.
The study — which involved researchers from KCL, the Norwegian University of Science and Technology, and the Abdus Salam International Centre for Theoretical Physics in Italy — was published May 14 in the journal Physical Review Letters.
While AI model collapse hasn't happened in a real-world scenario with an actively deployed AI system, anyone who uses tools like ChatGPT or Gemini to generate answers or text has very likely experienced errors or hallucinations. However, Roudi hopes the new findings might outline a method to sidestep this potential emergent threat.
Countering collapse
Beyond widely known hallucinations in primitive generative AI products, we may not have yet seen any dramatic examples of model collapse in the form of sophisticated AIs seemingly "going mad" and outputting complete nonsense. But signs of minor collapse could be observed when AI delivers increasingly inaccurate or bland answers to queries, or completely fabricates information while trying to generate some kind of output it assumes a user desires.
Sign up for the Live Science daily newsletter nowContact me with news and offers from other Future brandsReceive email from us on behalf of our trusted partners or sponsorsBy repeatedly training LLMs on data generated by other LLMs, the core truth and source of information — and spikes of variance between generations of models — get "smoothed out," delivering homogenized answers and outputs. For example, text that might read well enough at first glance could lack any real detail or nuance. Essentially, model collapse can be split into ‘early’ and ‘late’ stages, where the former sees an AI lose the ability to serve up edge-case (rare and or less common) information and produce bland, synthetic-feeling responses, and the latter sees LLMs deliver gibberish information.
The huge scale of LLMs and the data they process can make it hard to establish how and why they hallucinate information, and how certain choices lead to model collapse.
To tackle this, the researchers used smaller models that belong to exponential families — a catch-all term for a number of probability distributions, like ascertaining the likely outcomes from random events. The bell curve is one such example, as is figuring out the chance that a coin flip will land on heads.
"By looking at analytically tractable models such as the exponential families, you can answer those 'why' and 'how' questions," Roudi said. "By that same logic, you can come up with ways to mitigate its dangerous effects, how those ways work, and ultimately apply them to real-life examples."
The researchers discovered that by introducing a single external human-made data point to a pool of synthetic data used by a model undergoing closed-loop training, whereby a new model is trained on data generated by a previous models, they avoided model collapse.
Roudi said one example could be an AI-based image or video classifier, whereby an LLM is trained on data that includes a real image correctly classified by a human, rather than AI-generated media or media classified by an AI.
Related stories- AI for breakup texts? How 'sycophantic' chatbots are messing with our ability to handle difficult social situations
- Scientists build specialist 'AGI processor' that they believe will power the next wave of AI agents
- The more advanced AI models get, the better they are at deceiving us — they even know when they're being tested
"In other words, this data point would be linked to a 'ground truth,' something we know undeniably to be true and independently verifiable," Roudi said.
The next step for Roudi and the researchers is to apply this approach to larger and more complex models to see if this principle still holds true. This method could mitigate potentially "disastrous" scenarios of model collapse, especially within the AI models we use in everyday life, the team said.
"This research is the first step in setting out some ground rules for preventing this [from] happening in the future," Roudi concluded. "While more work should be done, AI engineers making things like the next ChatGPT can use what we've found to develop models that don't collapse."
Article SourcesJangjoo, F., Di Sarra, G., Marsili, M., & Roudi, Y. (2026). Lost in Retraining: Closed-Loop learning and model collapse in exponential families. Physical Review Letters, 136(19). https://doi.org/10.1103/156q-3ngc
Roland Moore-ColyerRoland Moore-Colyer is a freelance writer for Live Science and managing editor at consumer tech publication TechRadar, running the Mobile Computing vertical. At TechRadar, one of the U.K. and U.S.’ largest consumer technology websites, he focuses on smartphones and tablets. But beyond that, he taps into more than a decade of writing experience to bring people stories that cover electric vehicles (EVs), the evolution and practical use of artificial intelligence (AI), mixed reality products and use cases, and the evolution of computing both on a macro level and from a consumer angle.
View MoreYou must confirm your public display name before commenting
Please logout and then login again, you will then be prompted to enter your display name.
Logout
