The Myth of the Overnight Intelligence Explosion

Whenever a lab chief mentions recursive self-improvement, a particular image tends to form in the public mind: an AI that one night starts editing its own code, gets a little smarter, uses that to get smarter still, and by morning has become something no human can understand or stop. When Google DeepMind's Demis Hassabis recently named recursive self-improvement as a milestone the leading labs are chasing, that image resurfaced in earnest. We covered his remarks separately. Here we want to look at the scenario itself, because it is treated as a near-certainty far more often than the evidence warrants.

The idea has a name in AI circles. It is sometimes called "foom," or a hard takeoff, and the classic version comes from a long-running debate between the researchers Eliezer Yudkowsky and Robin Hanson. Yudkowsky argued for sudden, local, runaway improvement: one system bootstraps itself to vastly superhuman ability in a compressed window, days or weeks, before anyone can react. Hanson expected something far more gradual and spread across the whole economy. Both expected big change. They disagreed sharply on its speed and shape.

The case against the lightning bolt

A lot of serious researchers think the overnight version is simply wrong. The AI pioneer Francois Chollet laid out one of the better-known rebuttals in his essay on the implausibility of intelligence explosion. His argument, roughly, is that intelligence is not a single dial you can turn up in isolation. It is bound to the world it operates in: data, tools, experiments, other people, physical feedback. A system cannot think its way to mastery of biology or chip design without running real experiments in real time, and those have their own pace that no amount of cleverness erases.

There is also a plainer bottleneck: compute. A model that wants to train a smarter successor needs hardware to do it on, and hardware is physical, finite, and slow to build. A 2025 analysis argued that compute limits alone could blunt any sudden explosion, because the loop of self-improvement still has to wait on data centers that take years and billions to stand up. Dell's order book, which we wrote about this week, is a reminder of just how much steel and power a single generational leap consumes.

What is actually worth taking seriously

None of this means the worry is silly. The honest position, held by many of the people closest to the work, is that we do not have good tools to predict the shape of recursive capability gain. A "soft takeoff," where systems improve steadily and humans have time to watch and adjust, is plausible. So is a faster curve that is still measured in years rather than hours. What very few credible researchers defend is the literal overnight god-in-a-box, and yet that is the version that dominates headlines and dinner-table arguments.

So treat the lightning-bolt story as what it is: a thought experiment, not a forecast. The thing to track is not whether an AI "wakes up" some Tuesday. It is the slope of the curve, measured over months and quarters, in plain sight. That is a problem you can govern, audit, and slow down if you need to. The overnight explosion is mostly a way of telling ourselves the decision has already been made for us. It has not.