Will AI Slowly Poison Itself?

Here is a tidy piece of doom that has spread fast. The internet is filling up with AI generated text, the next generation of models will train on that sludge, and the whole enterprise will slowly rot into gibberish. People call it model collapse, or less politely, AI inbreeding. The lawsuit two American newspapers just filed against OpenAI reaches for the same image, warning of "a snake eating its own tail." It is a vivid fear. It is also only half right.

Where the fear comes from

The idea is not invented. In 2024 a team led by Ilia Shumailov published a paper in Nature with the blunt title "AI models collapse when trained on recursively generated data." They showed that if you train a model on the output of the previous model, then repeat that loop over and over, quality degrades. The rare cases at the edges of the data vanish first, the model's world narrows, and after enough generations the output turns to mush. The effect is real and it is measurable.

That result got compressed, as research often does, into a scarier headline: AI is running out of clean data and will eat itself alive.

What the fear leaves out

The catch is in how those experiments were run. Each generation was trained only on the previous model's output, with the original human data thrown away. That is not how anyone actually builds these systems. Real training sets accumulate. New synthetic data gets added to the existing pile of human writing rather than replacing it.

That distinction changes the outcome. A follow up study, Is Model Collapse Inevitable?, found that when synthetic data is accumulated alongside real data instead of swapped in for it, collapse does not happen. Since the real internet grows by adding new material rather than deleting the old, that accumulating picture is the realistic one. Labs also curate hard, filter aggressively, and increasingly use synthetic data on purpose, generating it deliberately to teach models specific skills. Done carefully, that improves models rather than breaking them.

So how worried should you be?

Model collapse is a genuine failure mode, not a hoax. If a lab were careless enough to train each model purely on the last one's output, the math is unforgiving. But that is a laboratory warning about a bad recipe, not a prophecy about the industry's future. The messier reality, where human and machine text pile up together and get sifted before use, is exactly the case where collapse mostly does not occur.

The newspapers' deeper point still stands, and it has nothing to do with statistics. If AI systems drain the outlets that produce reliable, original writing, the loss falls on readers and reporters, not on the models. That is a fear worth taking seriously. Whether the machines will poison themselves is not.