Does a Faster AI Mean a Smarter One?

Read the launch post for almost any new model this autumn and the first number you meet is a speed. So many tokens per second, so many times faster than the last release. It is an easy figure to grasp and an easy one to cheer, and it has quietly become shorthand for how good a model is. That shorthand is misleading.

Where the idea comes from

Speed is genuinely useful, which is why the confusion takes hold. A model that answers in a heartbeat feels cleverer than one that makes you wait, and faster generation does lower the cost of serving a product. New approaches lean into the impression. This week's Mercury 3 from Inception led with a claim of more than a thousand tokens per second, a headline built to turn heads.

What the number leaves out

The trouble is that throughput measures how quickly words come out, not whether they are the right ones. The two can move in opposite directions. When outside testers put Mercury 3 through a real coding task, they recorded a fraction of its advertised rate, because speed depends on the hardware, the batch size and the length of the request, none of which the headline mentions. Vendor benchmarks are run under flattering conditions, and quality claims in particular tend to sit unverified until independent labs catch up.

A model can also be fast and wrong. Generating a confident, fluent answer at great speed does nothing to guarantee the answer is correct, and a quicker route to a mistake is not progress. Reasoning, accuracy and the habit of not making things up are measured on different tests entirely, and a chart of tokens per second tells you nothing about any of them.

How to read the claims

This is a cousin of two misconceptions we have examined before, that topping a leaderboard crowns the best model and that a perfect test score signals a mind. Speed is simply the newest single number asked to carry more meaning than it holds. Treat it as one property among several. It belongs on the list next to accuracy, reliability and cost, not above them, and the only test that really counts is how a model does on the work you actually need it for.