Newer Claude Models Are Worse at Some Tool Calls

Here is a counterintuitive result from the past week. The newest and most capable models in Anthropic's Claude family, Opus 4.8 and Sonnet 5, are worse than their older siblings at using a particular software tool. The finding comes from the developer Armin Ronacher, who wrote it up on July 4, and it was quickly picked up by Simon Willison and others who work closely with these models.

The setup is ordinary. Ronacher builds an AI coding assistant called Pi, which gives the model an edit tool for changing files. To use it, the model has to produce a structured call that matches the tool's schema, a specific list of fields the code expects. Older Claude models did this cleanly. Opus 4.8 and Sonnet 5 kept inventing extra fields that were not part of the schema, so the program rejected the call and asked the model to try again.

The edits themselves were usually correct. The model knew what it wanted to do. It just insisted on describing the action in a format the tool did not accept.

A stronger prior can fight you

Ronacher's explanation is the interesting part. Anthropic's own coding agent, Claude Code, uses an edit tool with a simple shape: an old string, a new string, and an optional flag to replace every match. The newer models appear to have been trained so heavily on that pattern that they have formed a strong expectation of what an edit tool should look like. When a different program offers a tool with the same purpose but a different schema, the model quietly reverts to the shape it knows best.

In other words, the better-trained model fights harder for its own assumptions. A weaker prior bends to the schema in front of it. A stronger one keeps reaching for the pattern it learned in training, even when the tool on the table is asking for something else. Capability and obedience turn out not to be the same thing, and here they pull in opposite directions.

What to do about it

The practical fixes are undramatic. Ronacher found that turning on strict tool invocation, which forces the model's output to conform to the schema, made the problem disappear in his tests. Stripping earlier reasoning traces out of the conversation history cut the failure rate roughly in half. Aligning your tool's schema with the Claude Code convention, where you can, avoids the fight altogether.

The wider lesson is worth holding onto as more of the industry hands real work to autonomous agents. A model that has been optimised for one harness carries the habits of that harness with it. Those habits are usually helpful. Once in a while they are the thing getting in the way, and the only way to catch it is to watch what the model actually does rather than assume a smarter model must also be a better-behaved one.