OpenAI Built an AI Attacker to Break Its Own Models

Prompt injection is the security problem that has followed AI agents around since the moment they were given the ability to read a web page. Hide an instruction in a document, a calendar invite, or a page the agent visits, and the agent may follow it instead of the user. Every lab has acknowledged the issue. None has solved it, and the standard defence has been human red teams, which are slow, expensive, and limited by how many attacks a person can think of in a week.

On 16 July, OpenAI described its answer: an internal model called GPT-Red whose only job is to attack other models.

How it works

GPT-Red behaves like a human red teamer, just faster. It sends a prompt, watches what the target model does, and iterates toward a goal it has been given, such as getting the target to send sensitive data to an external server. It keeps adjusting until it either succeeds or exhausts its attempts.

The training method is what makes it interesting. OpenAI used self-play reinforcement learning, running GPT-Red against a pool of defender models and training both sides at once. The attacker is rewarded for producing a genuine failure. The defenders are rewarded for resisting while still completing whatever legitimate task they were given, which is the constraint that stops the defenders from learning to simply refuse everything. Neither side gets a fixed opponent, so both keep having to find new ground.

The numbers

OpenAI reports that GPT-Red succeeded in 84 percent of indirect prompt injection scenarios against GPT-5.1, on test cases held outside its training data. Human red teamers working the same set succeeded on a small fraction of it, reported at around 13 percent. That is a wide enough gap to be worth pausing on.

The company then used GPT-Red's attacks as adversarial training data for GPT-5.6. On its own benchmarks, GPT-5.6 Sol shows roughly six times fewer failures against direct prompt injection than the best model of four months earlier, and a failure rate of 0.05 percent against GPT-Red's direct attacks. One specific technique, a "fake chain-of-thought" attack that worked against GPT-5.1 about 95 percent of the time, now succeeds under 10 percent of the time. Help Net Security and The Hacker News both carried the figures.

What to hold back on

Every number above is OpenAI's, measured on OpenAI's benchmarks, against attacks generated by OpenAI's own attacker. That last part is the awkward one. A model trained specifically to resist GPT-Red will score well against GPT-Red. It tells you considerably less about attacks GPT-Red never learned to produce, and the whole history of adversarial machine learning is defences that generalise worse than their evaluation suggests.

OpenAI says a pre-print with more detail is coming, which is the thing to wait for. Independent evaluation on attacks from outside the lab is what would turn these numbers into evidence.

There is also a question nobody in the coverage seems keen to raise. A model that can autonomously discover working prompt injections against frontier systems is a capable offensive tool. OpenAI is keeping it internal, which is the obvious call, but the technique is now public and self-play against defender models is not a secret recipe. Whoever builds the second one will not necessarily share OpenAI's deployment policy.

Still, the direction is right. Defending against automated attacks with a human team was never going to scale, and prompt injection has been stuck for long enough that a genuine improvement is worth noting even with the caveats attached. We have written before about how agents still need supervision. Better injection resistance does not remove that, but it does raise the floor.