What a Token Is, and Why It Shows Up on Your Bill

Spend any time near AI pricing and one word keeps turning up. Tokens. Models are billed by the token, context windows are measured in tokens, and companies now lean on staff to spend fewer of them. It sounds like jargon invented to make a simple thing sound technical. It is not. A token is the actual unit these systems think and charge in, and understanding it clears up a lot at once.

Not words, exactly

A language model does not read words the way you do. Before it can process your text, the text is chopped into tokens, small chunks that are often parts of words. A rough rule of thumb is that a token runs about four characters in English, so a common word might be one token while a longer or unusual one splits into several. The phrase you are reading now is a handful of tokens, not a handful of words. This splitting is called tokenization, and it is the first thing that happens to anything you type and the last thing that happens before you get a reply.

Every token then becomes a set of numbers the model can do arithmetic on, because underneath the conversation a language model is a very large calculator. It does not manipulate language. It manipulates numbers that stand in for language. Tokens are the bridge between your sentence and its maths.

Why each one costs

Here is where the billing comes from. Processing a token takes computation, and computation takes chips, power and time. So providers price their models per token, usually per million, and they split the price two ways. Input tokens, the text you send in, are cheaper. Output tokens, the text the model generates, cost more, often several times more, because the model has to produce them one at a time in sequence. A pricing page that reads two dollars fifty per million in and ten dollars per million out is describing exactly that gap.

This is why a long conversation quietly gets more expensive as it goes. Most chat systems feed the whole history back into the model on every turn so it remembers what you said, and all of that counts as input tokens each time. The bill is not just your latest question. It is your latest question plus everything before it, over and over. It also explains the corporate belt-tightening, such as Microsoft telling its own engineers to ease off the tokens, and the steady stream of price cuts as labs compete on cost per token.

The quirks tokens leave behind

Once you see tokens, some of AI's odd habits make sense. Models have historically stumbled on tasks like counting the letters in a word, because they never see the letters, only the token the word became. Languages that tokenize less efficiently can cost more to process for the same meaning. And the hard limit on how much a model can consider at once, its context window, is a token budget, not a word count.

None of this requires you to think in tokens day to day. But the next time you see a price measured in them, or a model does something strangely clever or strangely dim, the token is usually the reason. It is the atom of the whole enterprise, and almost nobody outside the field has heard of it.