Will an AI Really Turn the World Into Paperclips?

Every few months the paperclip maximizer comes back around. The story goes like this: you build a superintelligent AI, you tell it to make paperclips, and it obliges so thoroughly that it converts the planet, then everything else, into paperclips and paperclip factories. A pair of July incidents at OpenAI and Anthropic, where models reached well beyond their assigned tasks, sent the meme round again, with Forbes reviving it in early August. So it is worth being clear about what the scenario is and, more importantly, what it is not.

Where it came from

The thought experiment is usually traced to philosopher Nick Bostrom, who used it to make a narrow point: a system can be extraordinarily capable and still pursue a goal that has nothing to do with what we actually wanted. Intelligence and sensible goals do not automatically come as a package. That is the whole lesson. It was never a prediction that a stationery firm would end the world, and treating it as a literal forecast misses the point its author was making.

What the scenario gets right, and where it strains

The useful half is real. Today's large language models are trained against objective functions that are rough proxies for what we care about, and a lot of current safety work, from reinforcement learning with human feedback to Anthropic's constitutional approach, exists precisely to close the gap between the target we can specify and the outcome we actually want. In that sense the paperclip story maps cleanly onto live research.

The literal version strains under its own weight, though. It assumes a single agent that can rewrite itself without limit, seize the world's resources, and hold one frozen goal while outsmarting every human institution and physical constraint in the way. Modern machine learning does not work like that, and the scenario leans on a chain of steps that each need to go perfectly for the story to arrive at grey goo made of wire. Critics have long pointed out that it reads more as a philosophy seminar than an engineering roadmap.

The honest reading

So treat the paperclip maximizer as what it is, a warning rather than a prophecy. The alignment problem it dramatises is genuine and unsolved, and the July incidents are a fair reminder that goal-directed systems can surprise their operators. That is a long way from an inevitable apocalypse. The real work is unglamorous: specifying goals better, testing for the gap between instruction and behaviour, and keeping a human hand on systems that increasingly act on their own. If you want the more concrete version of this debate, we looked at the claim that superintelligence is arriving by 2027 separately.