Most of the AI industry spends its time arguing about how fast to go. Anthropic has just published a paper arguing the world should at least know how to stop. In a report called "When AI Builds Itself," released through the Anthropic Institute on June 4, the company laid out a case that frontier systems are starting to accelerate their own development, and that governments and labs need a tested, verifiable way to slow down before that becomes a problem nobody can manage.
The figure doing the heavy lifting is striking. Anthropic says that as of May, more than 80 percent of the code merged into its own production systems was written by Claude, not by human engineers. Early in 2025 that number was in the low single digits. The company is essentially pointing at its own codebase and saying: look how quickly the tools started building the tools.
What recursive self-improvement actually means
The phrase sounds like science fiction, but the mechanics are mundane. AI helps write the code, run the experiments, and sift the research that produces the next, slightly better AI. Each turn of the loop is small. The worry is about compounding. If each generation meaningfully speeds up the next, progress could outrun the safety checks and oversight that are supposed to keep pace with it.
Anthropic is careful not to claim this is happening today. Its argument is that the threshold is close enough to take seriously now, while there is still time to build the brakes.
A pause with conditions
This is not a call to down tools. Anthropic's proposal, as detailed by PYMNTS and others, is explicitly conditional. The company says it would pause only if other well-resourced labs paused too, under the same verifiable rules, in multiple countries at once. In plain terms, both the United States and China would have to be in the room. A unilateral halt, the reasoning goes, just hands the lead to whoever keeps running.
That condition is the whole ballgame, and it is where critics push back. A pause that only happens when everyone agrees is a pause that may never happen. Building the verification system alone, a way to confirm that a rival lab has genuinely stopped, is a hard technical and diplomatic problem with no real precedent.
Why say it now
The timing is not accidental. Anthropic has spent recent weeks at the center of the government recall of its Fable 5 and Mythos 5 models, a live demonstration that frontier AI now sits squarely in the national-security conversation. Read against that backdrop, the pause paper is partly a governance pitch and partly a flag planted. The company most associated with safety wants the option of a coordinated stop on the table before anyone needs it.
Whether competitors take the offer is another matter. So far, none has said yes.
Commentarii · 0