Does AI Really Make Every Developer Faster?

With Meta, OpenAI and Anthropic now selling coding agents to anyone who will listen, one claim gets repeated so often it has hardened into common sense: give a developer an AI assistant and the work gets much faster. It is a tidy story. The trouble is that the best evidence we have does not quite support it.

The most talked-about study comes from Model Evaluation and Threat Research (METR), which ran a randomised controlled trial with experienced open-source developers working on their own projects. When the AI tools were switched on, the developers took about 19 percent longer to finish their tasks. The striking part is what those same developers believed. On average they estimated the tools had sped them up by roughly 20 percent. The gap between how fast the work felt and how fast it actually went is the whole point.

The productivity paradox

METR is not an outlier. Pull together the wider research from DORA, Bain, GitHub and others and the results scatter wildly, from around 26 percent faster in some settings to 19 percent slower in others. At the level of a whole company, many organisations report no measurable productivity gain at all, even as nearly every developer uses the tools. That pattern has a name now, the AI productivity paradox, and it describes the distance between obvious individual enthusiasm and stubbornly flat aggregate numbers.

Look closer and the effect depends heavily on the task. AI assistants do well on narrow, self-contained problems, the kind of isolated code generation where studies have found real time savings. They struggle more with the work that fills an experienced engineer's day: navigating a large existing codebase, debugging something subtle, or making architectural decisions where being confidently wrong is expensive. A tool that writes a plausible function in seconds can still cost you an hour when the function is plausible and wrong.

Where the myth breaks down, in both directions

This is not a case for throwing the tools out. Honesty cuts the other way too. In February 2026, METR revisited its own experiment design and acknowledged real limitations, including that the developers who benefit most from AI often declined to take part in sessions where they had to work without it. So the 19 percent figure is not proof that AI makes programming slower. It is evidence that the picture is mixed, and that the confident marketing numbers are built on shakier ground than they appear.

The reasonable reading is the boring one. AI coding tools clearly help in some places, especially for boilerplate and unfamiliar syntax, and clearly do not deliver a uniform speed boost across every developer and every task. Feeling faster is not the same as being faster, and the people best placed to measure the difference are the ones least likely to notice it in the moment. The myth is not that the tools are useless. It is that the benefit is universal, large and automatic. On current evidence, it is none of those three by default.