You hear it in every comment thread about AI, usually delivered as the line that ends the discussion. A large language model is just autocomplete. It is only guessing the next word, one at a time, so whatever it seems to be doing, nothing is really going on in there. The claim is popular because the first half is correct. The trouble is the conclusion people hang on it.
Start with what is true. At the mechanical level, a model like Claude or GPT does exactly one thing: given a stretch of text, it produces a probability for what token comes next, picks one, and repeats. It learned to do this by being shown an enormous amount of writing and adjusting itself, over and over, to predict the missing piece. There is no hidden second system. The whole machine is a next-token predictor, and anyone who tells you otherwise is overselling it.
Where the dismissal falls apart
The leap people make is from how it works to what it can do, and that leap does not hold. Predicting the next word well enough, across billions of examples, turns out to require learning a great deal about the structure behind the words. To guess the end of a sentence about gravity, it helps to have absorbed something about how objects fall. To finish a line of code, it helps to have internalised the rules of the language. Researchers who probe inside these models keep finding organised internal representations, rough maps of grammar, geography, even the state of a game being described in text. The simple objective produced structure nobody wrote down.
That is the part the autocomplete line misses. The phone keyboard that finishes your texts is also next-token prediction, and it will never explain a proof or debug a function. Scale and training changed what the same basic trick is capable of. Calling both of them autocomplete is like calling a hospital and a first-aid kit the same thing because both involve bandages.
But the other direction oversteps too
None of this means the model understands in the way you do, and this is where the opposite camp gets carried away. A system can build a useful internal model of the world without having beliefs, intentions or any awareness of what it is saying. It still produces confident nonsense when the statistics lead it astray, which is why it sometimes reproduces its training data and why hallucination remains a live problem. And there is no reason to think the lights are on behind the text, a point we examined when we asked whether chatbots might be conscious.
So the honest answer sits between the two slogans. Yes, it is predicting the next word. No, that does not make it trivial, and no, that does not make it a mind. The mechanism is genuinely simple. What grew out of it is not. Holding both of those facts at once is harder than reaching for either catchphrase, which is probably why so few people bother.
Commentarii · 0