Does an AI Keep a Copy of Everything It Read?

Every fresh copyright suit against an AI company tends to revive the same mental picture: a model as a giant filing cabinet, holding a copy of every book, song and web page it was trained on, ready to hand any of them back on request. The lawsuit over Elvis Presley's lyrics leans on exactly that image. It is worth asking how true it is.

Mostly, it is not. A large language model does not store the text it learned from. Training adjusts billions of numerical weights so the model gets better at predicting the next word, and the original documents are not saved anywhere inside it. A model trained on terabytes of text can itself be a few gigabytes. The library does not fit in the cabinet, because the model never kept the library.

So why does it sometimes quote things word for word?

Here is the catch that keeps the myth alive. Models can and do reproduce chunks of their training data verbatim, a behaviour researchers call memorisation. It happens most with text the model saw many times over, popular song lyrics, famous passages, widely copied code. Something repeated thousands of times across the training set leaves a deep enough groove in the weights that the model can retrace it.

That is why the two sides of a case like this can both sound convincing. The AI company can say, truthfully, that it did not store a copy of the song. The rights holder can say, also truthfully, that the model will nonetheless produce the lyrics on demand. Both can be right at once, which is precisely what makes the law here so unsettled.

What this means for the fear

The useful way to think about it is probability, not storage. A model has learned the statistical shape of its training data so thoroughly that for heavily repeated text, the most likely next word is simply the real next word, over and over, until it has rebuilt the original. For the vast majority of what a model learned, seen once or twice and never again, nothing recoverable survives.

None of this settles whether training on copyrighted work is lawful. That is a question for the courts, and they are working through it case by case. But the engineering underneath the argument is not a mystery, and it is not a filing cabinet. It is a very large set of probabilities that, now and then, point so sharply in one direction that the original text falls out intact.