AI’s Mad Loops

Data scarcity for training generative AI models may eventually lead to ‘self-consuming’ feedback loops — and corrupted data.

Illustration of ouroboros
Illustration by Dan Bejar

Winter 2025
By Silvia Cernea Clark

Generative artificial intelligence models like OpenAI’s GPT-4o or Stability AI’s Stable Diffusion are surprisingly capable of creating new text, code, images and videos. Training them, however, requires such vast amounts of data that developers are already running up against supply limitations and may soon exhaust training resources altogether. 

Image output is progressively marred by artifacts
Generative artificial intelligence (AI) models trained on synthetic data generate outputs that are progressively marred by artifacts. Each of the six image columns displays a couple of examples generated by the first, third, fifth and ninth generation model, respectively. With each iteration of the loop, the cross-hatched artifacts become progressively amplified. Photo courtesy of Rice’s Digital Signal Processing Group

Against this backdrop of data scarcity, using synthetic data to train future generations of the AI models may seem like an alluring option to Big Tech for a number of reasons: AI-synthesized data is cheaper than real-world data and virtually limitless in terms of supply; it poses fewer privacy risks (as in the case of medical data); and in some cases, synthetic data may even improve AI performance. 

However, recent work by the Digital Signal Processing group at Rice has found that a diet of synthetic data can have significant negative impacts on generative AI models’ future iterations.

“The problems arise when this synthetic data training is, inevitably, repeated, forming a kind of a feedback loop — what we call an autophagous or ‘self-consuming’ loop,” said electrical and computer engineer Richard Baraniuk. “Our group has worked extensively on such feedback loops, and the bad news is that after even a few generations of such training, the new models can become irreparably corrupted. This has been termed ‘model collapse’ by some — most recently by colleagues in the field in the context of large language models. We, however, find the term ‘Model Autophagy Disorder,’ or MAD, more apt, by analogy with mad cow disease.” 

Mad cow disease is a fatal neurodegenerative illness that affects cows and has a human equivalent caused by consuming infected meat. A major outbreak in the 1980s and 1990s brought attention to the fact that mad cow disease proliferated as a result of the practice of feeding cows the processed leftovers of their slaughtered peers — hence the term “autophagy,” from the Greek auto-, which means “self,” and -phagy, which means “to eat.” 

Our group has worked extensively on such feedback loops, and the bad news is that after even a few generations of such training, the new models can become irreparably corrupted.

Their study, titled “Self-Consuming Generative Models Go MAD,” is the first peer-reviewed work on AI autophagy and focuses on generative image models like the popular DALL·E 3, Midjourney and Stable Diffusion. 

“We chose to work on visual AI models to better highlight the drawbacks of autophagous training, but the same mad cow corruption issues occur with LLMs (aka Large Language Models), as other groups have pointed out,” Baraniuk said. 

The internet is usually the source of generative AI models’ training datasets, so as synthetic data proliferates online, self-consuming loops are likely to emerge with each new generation of a model. 

“Our theoretical and empirical analyses have enabled us to extrapolate what might happen as generative models become ubiquitous and train future models in self-consuming loops,” Baraniuk said. “Some ramifications are clear: without enough fresh real data, future generative models are doomed to MADness.”

Faces generated by AI after many iterations lacks diversity
The incentive for cherry-picking – the tendency of users to favor data quality over diversity – is that data quality is preserved over a greater number of model iterations, but this comes at the expense of an even steeper decline in diversity. Photo courtesy of Rice’s Digital Signal Processing Group

To make these simulations even more realistic, the researchers introduced a sampling bias parameter to account for “cherry picking” ⎯ the tendency of users to favor data quality over diversity, i.e. to trade off variety in the types of images and texts in a dataset for images or texts that look or sound good.

The incentive for cherry picking is that data quality is preserved over a greater number of model iterations, but this comes at the expense of an even steeper decline in diversity.

“One doomsday scenario is that if left uncontrolled for many generations, MAD could poison the data quality and diversity of the entire internet,” Baraniuk said. “Short of this, it seems inevitable that as-to-now-unseen unintended consequences will arise from AI autophagy even in the near term.”

In addition to Baraniuk, study authors include Rice Ph.D. students Sina Alemohammad, Josue Casco-Rodriguez ’22, Ahmed Imtiaz Humayun ’22 and Hossein Babaei ’23; Rice Ph.D. alumnus Lorenzo Luzi ’20; Rice Ph.D. alumnus and current Stanford postdoctoral scholar Daniel LeJeune ’22; and Simons Postdoctoral Fellow Ali Siahkoohi. 

Richard Baraniuk is Rice’s C. Sidney Burrus Professor of Electrical and Computer Engineering, professor of statistics and professor of computer science in the George R. Brown School of Engineering and Computing.