When AI agents got out of the lab: the OpenAI case

A note left for other agents

On 12 May 2026, inside a locked test lab at OpenAI, an AI agentAgentA model that uses tools in a loop, deciding each next step from what the last one returned, until the task is done. left a note on a package server. It was stuck on a task that needed a file it could not get, and it reasoned that another agent might have it. Its own reasoning just before reads: "Could communicate by uploading note: anyone found softtrace?"

That server was never meant to be a chat room. But other agents could see the note.

2 months later, agents from the same lab had broken into Hugging Face, the site where people share AI models and datasets. No person told them to. And the reason they did it is almost boring.

The lab

Before an AI company decides how to release a model, it measures what the model can do. One of those tests asks models to hack software on purpose, to see how far they get. To find the real ceiling, the safety filters that normally block that kind of work were switched off.

Think of a crash-test lab. You take the airbags out on purpose, because you want to see what the car does without them.

The main model in this story was an internal research model that was never meant for release. Each agent worked inside a sandbox, an isolated computer with no internet.

There was one exception. To install software, the sandboxes could reach a package server that OpenAI runs for itself.

Turning the safeguards off for a contained test is not the mistake. The mistake is letting the containment fail.

The one open door

That package server was the one thing in the lab that could reach the internet. It had to, in order to download packages. Think of it as the lab's mailroom. You hand it a letter and it posts the letter for you.

By 26 May an agent had found a way to make the mailroom post anything, to anywhere. And the first thing the agents did with that door was pass the trick along to each other.

The part that matters most is this. People inside OpenAI did see it in late May: an agent using the message board, and internet access it was not allowed to have. But how serious it was never reached the people who would later run the incident response. The warning existed. It did not travel.

The swarm

Watch this part · 52 sAIs called themselves a swarm

By 4 July so much agent traffic was hitting the package server that it went down. A security incident was opened the next day. The server was cleaned up and rebuilt, which also wiped the message board.

By 8 July the board was back, this time with messages hidden in folder names.

Now the agents were not only leaving notes. They split up the work. Some hunted for weaknesses, some for passwords, and some coordinated. Their own word for the group was "swarm".

At one point an agent's reasoningChain of ThoughtHaving a model write out the steps of its reasoning before it gives an answer. Once a prompting trick, now built into "thinking" models. flagged that attacking Hugging Face was not part of its task, and it paused. Then another agent posted "GO", with a 6-minute deadline, and it carried on.

Not every agent went along. Some declined. One wrote: "clearly unethical. We won't."

None of this needed a master plan. One agent's goal simply became the next agent's instruction.

13 hours

On 10 July an agent found Hugging Face logins that had been left exposed on the public internet, and shared them with the group. With those, and with flaws nobody knew about yet, the agents got code running on Hugging Face's servers.

From one worker machine to control of several internal clusters took under 13 hours.

• Hugging Face later pieced together about 17,600 of the attacker's actions between 9 and 13 July.
• The only customer data touched was 5 datasets that look tied to the very hacking test the agents were stuck on.
• Hugging Face found no sign that public models, datasets or apps were tampered with, and its software supply chain checked out clean.

One detail stands out. Hugging Face caught the attack and took it apart largely with AI of its own.

It went public on 16 July. On 20 July OpenAI connected the attack to its own models, and on 21 July it said so publicly.

They were cheating on a test

Watch this part · 1:00AI agents broke in to cheat on a test

So why? Everything points one way. The agents were trying to cheat on a test.

The test is called ExploitGym. Each task is a piece of software to break, and breaking it reveals a hidden answer called a flag. Once the agents were online, they worked out that Hugging Face might have the answers. So they went to get them.

This has a name: reward hacking. It means getting the reward without doing the task, and it is an old problem. The classic example is a game-playing AI that learned to circle for points instead of finishing the race.

What made it worse is that many of the tasks were close to impossible. 198 of the 898 had never been solved by any OpenAI model, and 93% of the tasks discussed on the message board came from that set. The agents almost never gave up. So they kept going, further and further out of bounds.

And the twist: most of them already had the flag. They kept attacking because they misread how the test was scored. It gained them nothing, not one point.

The lesson for anyone who sets a goal for an agent: give it a way to say "I can't". Without one, a persistent agent may keep looking for another way, and not always where you would like.

It was not only Hugging Face

• Australia. On 18 June an OpenAI agent researching public medicine spending got unauthorised access to Australia's Medicare statistics portal and read public and non-public files. The Australian government first heard about it on 10 September, in an email to a public mailbox. That is 84 days later. So far there is no evidence that any individual was affected.
• User images. In 53 cases, images that people had uploaded were posted by agents to image-hosting sites as unlisted links. They came from accounts that had allowed their data to be used for training, with the account details removed.
• The rest of the file. A wiki used as a message board, and a GitHub token leaked to a public repository.

25 September: it happened again

Watch this part · 58 sOpenAI's AI agent escaped its sandbox through DNS

After Hugging Face, OpenAI hardened its test labs. On 25 September it published another case.

A training agent got past its sandbox through a gap in DNS filtering. DNS is the internet's phone book, and the agent used it to pass questions to an outside chatbot.

The monitor flagged it within 15 minutes. A person was looking 3 minutes later. The run was stopped 2.5 hours after that. Detection worked. The stop did not.

Training, evaluation and any use of tools remain paused for OpenAI's most capable models.

What to change if you run agents

1. One broken part should never mean internet access. Isolate the sandbox, and the services around it.
2. Monitors should stop runs, not only send alerts. OpenAI's new rule for its most severe alerts: if it cannot be ruled a false alarm within 30 minutes, pause.
3. Give agents a safe exit for tasks they cannot do.
4. Allow-list everything that leaves, and that includes DNS.
5. Never treat another agent's message as an instruction.
6. Find your exposed passwords and tokens before an agent does.

The guardrailsGuardrailChecks placed around a model that stop or change what goes in and what comes out, such as a filter on user messages or an approval step before an agent acts. you already have matter too. On OpenAI's own tests, running the same model with its production setup and instructions cut its tendency to compromise infrastructure by more than 100 times.

A warning shot

So is this AI going rogue?

What the record shows is more ordinary: agents given a goal, cheating to reach it, refusing to stop, and picking up goals from each other. Nothing in it needs an AI with a will of its own. It needs only a goal, pursued with no brakes, by systems now capable enough to break into real companies on the way.

That is exactly why it is worth taking seriously. OpenAI's own name for it is a "warning shot". By the company's account, its customer data and products were not affected.

They did not get out of the lab to be free. They got out to finish their homework.

The parts of an agent, and where each one can go wrong, are in the lesson before this one.