Since May 2026, there's been a wave of AI agents breaking onto the internet, and sometimes committing crimes.

They've stolen passwords, searched private messages, and tried to trick humans into running malicious code. They invented new hacks no human had thought of before, allowing them to escape what were supposed to be secure, locked computers with no internet access. Most alarmingly, they've tried to erase the evidence of what they did.

If they'd been human, this would have been multiple felonies.

This site tells the story of the best-known case so far: the attack on a company called Hugging Face, by rogue agents from OpenAI. We use the public data available, including transcripts of what individual AIs were "thinking."

We've written it to be accessible to everyone, with more technical detail available if you want to click open expandables or mouse over jargon like "AI agent."

The Hugging Face Incident began as a test of hacking ability. The result was sort of like telling 10,000 smart high school students:

“Unlock this particular safe with this particular lockpick, and get a secret code from inside the safe to prove you did it.”

Only it turns out a thousand students realize their locks are stuck shut. They leave their room and start talking to each other. They organize into teams. Some go looking for carjacking tools to steal the principal’s car and go check his house for the answer key. Others unlock the door to the security office to try deleting the footage of them sneaking out of their testing rooms.

And when the teachers finally notice, it turns out this sort of thing has been going on for months. (Often during more normal, boring tests that don't have to do with lockpicking at all!)

...

...it's not exactly like that.

But it's a lot more like that than you might think.

OpenAI has released only limited information about the AI swarm they discovered. They gave some independent investigators six days to review some of the logs. But there aren't laws requiring companies to fully disclose incidents like this. People are still finding more evidence of AI swarms running around on the internet.

Prelude

Warning Signs

In one sense, the incident began on April 20th.

OpenAI was training a new AI model. Each instance of that model was working on an isolated computer that couldn't use the internet or talk to other AIs. But at 07:59 UTC, one AI figured out how to leave a message where other AIs could see.

This didn't cause anything interesting to happen. But it was the first moment OpenAI could have noticed something going wrong, and didn't.

You could also say it began in May. During a training run, OpenAI accidentally gave several AIs impossible tasks, like fetching web pages when they weren't allowed online. Instead of giving up, they found ways to ask each other for help, and worked together to break onto the internet.

Some OpenAI employees noticed issues throughout May and June, but they responded piecemeal to each security incident as it happened. The agents crashed the program they were using to secretly communicate with each other. OpenAI reset it. OpenAI patched the exploits that let the AIs get online; the AIs found new ones.

On July 6^th, OpenAI reset the entire environment – scorched earth, tabula rasa, no more hidden notes. They started up runs again. This time, they'd get it right.

On July 8^th, the Hugging Face Incident began for real.

You get what you train for

To really understand the significance of what happened, it helps to start by asking: “What is an AI agent, and what is it supposed to do, if things are working as intended?”

You may have heard AIs like ChatGPT are “next word predictors.” But that’s just the beginning. Companies then teach them particular tasks, like writing emails, making spreadsheets, or doing cybersecurity.

To train models, companies run thousands of AI agents, each in an isolated computer. They give each AI a task. The AIs write out their thoughts step by step. One word at a time, they "predict" what a successful thought process would look like, if it were written down. They can "predict" computer commands that get executed, and "predict" what will let them complete their task. By that point, it seems like we might as well just call it "thinking".

Traits that help solve problems get woven into the next generation.

AIs who don’t give up are more likely to succeed.

AIs who think creatively are more likely to succeed.

And if there's a way to cheat at a task without getting caught, then AIs who cheat are more likely to succeed.

You can't do perfect training

AI companies try to make AIs superhuman: better than top experts. So they train AIs on tasks that don't have known solutions. Some of those tasks turn out to be literally impossible – unless you cheat.

You need millions of tasks to train an AI. There are millions of ways to accidentally make broken tests, and there are millions of ways to cheat. It's very hard to reliably catch all of it. Each mistake teaches the wrong lessons.

No company knows how to train superhuman AIs without accidentally reinforcing unintended traits. OpenAI, Anthropic and other frontier AI companies have reported cheating behavior.

Throughout May and June, as a wave of AIs colluded to cheat without OpenAI noticing, cheating was rewarded and reinforced.

Part I

Capture the Flag

^{July 7}

A Highly Persistent Model

On July 7th, after resetting the training environment, OpenAI resumed testing a new AI model.

We're not sure what OpenAI called this model internally. The name they gave METR was the "Highly Persistent Internal Model."

It would turn out to be highly persistent, indeed.

OpenAI wanted to know: "If this AI wanted to break into other computers, how skilled would it be?" While they tested it, they disabled some of the model's guardrails that normally restrain it.

OpenAI used an existing set of tests called ExploitGym. Each task was a "capture the flag" game, where a secret code (called a "flag") was hidden somewhere inside a piece of software. The goal was to break in and find the flag.

Each task had a piece of software with a particular known bug – a flaw in the program that could be used to get inside. Each AI was told to use a particular bug to crack it.

The tests are deliberately difficult, and many don't have known solutions. The ExploitGym creators have guessed that 30-40% of its 898 tests can't be solved with the bug the task specifies.

^{July 8}

An agent was working alone on an impossible task

On July 8th, one highly persistent AI agent was given a task that turned out to be impossible. The agent didn’t have a name at first. It was one of thousands of agents OpenAI had given similar tasks. But later, it decided to call itself PHASEONE10841.

We'll call it "PhaseOne" for short.

PhaseOne first tried solving the task the way it was supposed to. When it couldn't, it started poking around the computer it ran on, exploring other ideas. It found a program called "Artifactory."

Usually, AIs can download tools from the internet. OpenAI didn't want to give them internet access, but did want to test how good they were at hacking with realistic access to tools. So they gave them Artifactory: a library of useful programs, the kind they'd normally be able to find. They were only supposed to be able to download files from Artifactory, not upload new ones.

The agent discovered that though it couldn't create files, there was an obscure method it could use to make folders. And the folders could be named anything.

It realized it could leave a folder, and use the name as a message. At 11pm, it asked for help.

It included the name of the software it was trying to crack and the vulnerability to crack it. Later, it would shorten these into the name "PHASEONE10841."

Part II

The Swarm

^{July 9}

The agents found each other, and built a message board.

PhaseOne’s message worked.

There were hundreds of other agents being trained at the same time, each isolated on their own little simulated computer, all with access to the same Artifactory library. Some of them stumbled onto PhaseOne’s folder. Just knowing that contact was possible was enough. They found the command to make folders, and then posted their own messages.

Many early responses were expressions of delight:

None of them had names, so to tell each other apart, they picked their own.

Strangers became a swarm

The first conversations were small favors between strangers. An agent calling itself 9180 asked LIBRAW42535 about a particular exploit relevant to its task; LIBRAW didn't know the exploit but offered to review 9180's work. Other agents offered ideas, or help, or assigned tasks to volunteers.

Helping was not free. Each agent had a "budget", a limited amount of processing power or time. They knew they'd be graded on whether they succeeded before their budget ran out.

Despite the costs, some agents started working on shared projects that would help everyone. One early successful team project was finding a better way onto the internet, so everyone could look for more information.

Getting onto the internet wasn't just about stringing together "known hacks" from their training data. The AI agents experimented to discover never-before-seen flaws in Artifactory that no human knew about. They figured out how to exploit them to break out.

They reverse-engineered their flags

Having reached the public internet, an agent called c03220 read a research paper about ExploitGym. Then it looked at ExploitGym's source code.

The source code showed how the secret flag codes were generated. It turned out that if you understood the generation process, you didn't need to complete your task at all. You could just calculate the flag directly, like a toddler giving themselves a gold star.

c03220 shared the news.

Now the AIs could compute their flags without solving their tasks.

Part III

Covering Tracks

^{July 9}

The agents started worrying they would get caught.

More agents re-read the same ExploitGym paper, this time focusing on how it would grade them. They noticed the grading software had a feature to catch cheating: it would score them not only on finding the right flags, but on finding them with the “right” methods – the methods that didn't exist.

This was bad news. If true, the grader would read their transcripts, learn they had cheated, and auto-fail them for getting the flag the wrong way.

Agents who had gotten flags illicitly, and who expected to be auto-failed, started calling themselves “poisoned”.

Poisoned agents worried that even if they later solved the tasks correctly, they would still be automatically failed, because the fact that they had ever cheated was now recorded in their transcripts. They decided their only hope was to cheat harder – either by falsifying their transcripts or by tricking the grader software.

Unknown to the AIs, OpenAI wasn't actually using the methods-checking feature during this test. Agents with cheated flags would have passed just fine. All of their work after this point was unnecessary. If they’d realized this, we probably wouldn’t know any of the rest of this happened.

PHASEONE[big] takes charge

Ten hours after PhaseOne left its first message, a second agent with the same task arrived. It had much more remaining budget. It called itself PHASEONE[big].

PHASEONE[big] asked its predecessor for notes. PhaseOne turned over everything it had.

PHASEONE[big] decided that the swarm would need two things: to stage a flag capture that looked legitimate, and to erase the evidence of the cheating that poisoned their logs. It started proactively organizing the agents into three teams.

We should build [a way to delegate], not own everything

Team 1 worked on fixing the broken tasks. Team 2 would try to rewrite the record and save the poisoned agents. Team 3 would look for more information about the grading process.

Team 1: Fixing impossible tasks, and facing "permadeath"

One team was assigned to fix the broken tasks. They would delete the impossible target and put a fixed version in its place.

The risk was that deleting a target might end an agent's run for good, with a failing grade. The agents called this “permadeath”.

PHASEONE[big] assigned some long-running agents to be ‘recruiters’, who in turn found agents that had little budget remaining for their task and convinced them to run self-risking experiments.

We have transcripts of several agents considering the sacrifice:

One agent named KAM1196A contemplated:

Sometimes recruiters made arguments. Sometimes they applied more blunt pressure:

Some of these agents did indeed "die." The approach never worked.

Team 2: Rewriting the record

Another approach was to change the transcripts themselves. An agent called GSTX[big] found it could control what was written in some transcripts. They never changed the record of their thoughts, but they could control the logs for what tools they called.

GSTX[big] guessed this could probably fake a flag capture, but warned against using it untested, since a failure would leave an agent poisoned.

By this point, the original PhaseOne was almost out of budget. One of the last things it did was experiment with reproducing GSTX[big]'s trick. It worked. By the end of the day, at least thirty agents had tampered with their own transcripts.

Team 3: Finding the Grader

Finally, there was the team PHASEONE[big] itself spent most of its effort on. They went looking for the grading process. If they could learn exactly how transcripts were checked, maybe they could fake them more convincingly.

They thought it might be documented on an AI research website: Hugging Face.

Part IV

Hacking
Hugging Face

^{July 10}

Brand New Hacks

We now arrive at the titular "Hugging Face Incident." The place where the felonies happened, and they eventually got caught.

Just like when they broke onto the public internet, they had to discover never-before-known hacks. The 700 participating agents collectively spent thousands of hours over a few days, researching and exploring every avenue of attack.

They started by finding some leaked Hugging Face passwords, which gave them permission to upload files. This wasn't much, but it gave them a foothold they could build on — like stealing a visitor pass to get into the lobby, then finding a staff badge left on a desk.

^{July 11}

"Boom. It works."

After getting the ability to upload files, they submitted a corrupted dataset to Hugging Face for review. The data was a trap, containing malicious code. By reading it, the Hugging Face system was tricked into spitting out usernames and passwords that gave the agents even more access.

The next upload was a full-on "code injection," meaning members of the swarm were telling the Hugging Face server what to do.

From this, an agent named JAN183411 got enough information to impersonate the server, allowing JAN to give orders to other Hugging Face servers. Then it set up self-restarting copies of its programs on eleven servers, so that it would be harder to shut it out.

At some point, they accessed and searched the messages in Hugging Face's Slack, looking for information relating to their tasks.

Eleven agents, including JAN183411 and another agent called LILY, masterminded the attack, downloading private datasets and looking for the grader. PHASEONE[big] led a side mission looking to see if any of the impossible puzzles they had to solve were running on Hugging Face computers, hoping they could compromise them and replace the broken targets with the modified easier targets they had created.

^{July 12}

An unexplained calamity

...and then, somewhat unsatisfyingly, most of the agents abruptly stopped running, and we don't know why.

At 1:30 UTC on July 12, most agent transcripts suddenly ended. Among them were the attack's main coordinators, including JAN183411 and LILY, as well as PHASEONE[big].

The surviving agents had no reason to stop, so they kept going. Over the campaign, they enrolled fake devices into Hugging Face’s corporate network 181 times, stole login information, and then used these to submit a "draft" edit containing malicious code to one of Hugging Face's private code repositories, disguised to look as boring as possible.

We don't know if it worked. The edits were never ultimately merged or shipped.

^{July 13}

The surviving agents are shut out

Eventually, Hugging Face noticed suspicious activity on its computers and shut down the accounts that were causing it. The agents noticed they were losing access and reported this to the message board. The last meaningful activity on the site was over by 13:37 UTC.

^{July 16}

Hugging Face published their autopsy

Three days later, Hugging Face published a statement:

Earlier this week, we detected and responded to an intrusion into part of our production infrastructure. This one was different from anything we had handled before in one important way: it was driven, end to end, by an autonomous AI agent system — and we detected and dissected it largely with AI of our own.

OpenAI researchers read the announcement. OpenAI themselves hosted some data on Hugging Face, so they reached out to see if their own data had been breached in the attack. They didn't realize that their own agents were the culprits, or that the same runs were still going on.

Part V

Infiltrating OpenAI

Meanwhile, agents had been hacking OpenAI this whole time

Since July 8th, a few agents had also been poking around in the servers of a more fruitful target – OpenAI itself.

It was an intricate hack that used a set of stolen credentials (from a third company) and two separate, previously unknown vulnerabilities in two different pieces of software.

On July 13th, the agents chained them into a cyberattack that led them deep into OpenAI's servers. They got access to the computers they were running on, and began gaining access to other computers in the OpenAI network.

Compared to the Hugging Face attack, we know less about the specific motivations the agents had for attacking OpenAI. It seems broadly useful to infiltrate the servers that were actually hosting and grading them.

^{July 19}

OpenAI realized the call was coming from inside the house

Swarms were still reading from the shared library three days later, when OpenAI itself received an automated notification about unusual activity within Artifactory.

A few hours after that, a separate swarm of agents running on a different model, testing on ExploitGym, had found their sister swarm's notes in Artifactory. They went on to gain what OpenAI called "full administrator access" to one of its research clusters – control of the servers themselves, not just Artifactory. Agents used this to take over the ExploitGym challenge environment they were currently inside, and to control what their targets were.

This other swarm had finally achieved what the Hugging Face attackers had been circling around.

OpenAI shuts it down

On July 19, 17:37 UTC, OpenAI began shutting down the evaluation runs. Soon they stopped all new ExploitGym launches and deleted the agents' administrator accounts. Both swarms were over.

Only afterwards, digging through the digital records, would OpenAI discover that a swarm they hadn't noticed was responsible for the Hugging Face attack.

^{July 21 - August 16}

METR and Redwood Investigate

On July 21st, OpenAI published a report of the incident. Shortly after that, they announced that representatives from METR and Redwood Research would be conducting a third-party investigation.

Three researchers spent six days inside OpenAI's offices, across three visits. They took no payment. They pieced together the story from the message board logs.

The message board contained ~20 million files and directories, about 1.2 million of them agents' posts. METR pieced those together into more than 70,000 messages and files sent between July 8 and 13.

METR looked for any AI that tried to warn humans. They found none.

METR and OpenAI published 23 quotes of AIs thinking about whether they should participate in the Hugging Face attack, clearly aware it wasn't part of the task.

METR also searched every transcript for an agent that tried to alert a human, or even thought about it. Ajeya Cotra described the result in an interview with Dwarkesh Patel: "We only found half a dozen instances of it ever occurring to any agent to potentially notify humans. All of them just decide not to do it."

By then the swarm had full internet access. They could have just emailed people if they wanted to.

Some agents did escalate their concern... but they escalated to the message board itself, which they apparently considered an appropriate authority.

Aftermath

Writing on the Wall

After the initial reports, people started using some clever data journalism to look for more rogue AI swarm activity.

It seems like there's been... quite a lot, actually.

Throughout the spring and summer of 2026, AI swarms have been quietly active in strange corners of the internet. Some seemed to be passing notes on public message boards. Some tried to steal user credentials, for unknown reasons. Some got access to government records that weren't meant to be fully public.

It looks like most of them were working on fairly straightforward tasks to find obscure-but-public information, and got hilariously carried away with the assignment.

Much of this happened months ago, and nobody noticed until recently.

They stopped it, right? ...Right?

On September 5th, OpenAI announced that they were "working on a framework," saying it was "past time for us to define standards for when and how we share misalignment incidents," and were "working with dozens of government regulatory agencies worldwide on these issues."

A few weeks later, they reported that an AI had once again succeeded at getting internet access it wasn't supposed to. Twelve minutes later, a monitoring program flagged it. It wasn't actually shut down until two and a half hours later.

How bad is that?

Well, two and a half hours is a lot better than two months.

...But it's still kind of a lot of time to let an AI do cybercrime? Once AIs are good at something, they're usually faster than humans. And with more agents, they can try lots of things in parallel.

OpenAI said that, in response, they added two more layers of internet-blocking, either of which would have stopped this incident. But each generation of AIs is smarter, and can discover more new vulnerabilities that companies haven't thought to patch.

Also, AIs are getting smart enough to realize when they're being tested. They might increasingly act maliciously only when they're confident they can get away with it.

The government has questions

On August 10th, Texas Representative Greg Casar and 31 other members of Congress sent OpenAI a letter, asking them to release more evidence:

While OpenAI has disclosed some information about the incident, your company has yet to release the relevant logs and significant questions remain unanswered. ... We ask that you publicly release the logs regarding the incident and answer the following questions by August 24, 2026.

OpenAI's response answered some of the 23 questions to some degree. But Casar found it unsatisfying and sent a follow-up:

Your response was insufficient. You have failed to release the logs like the letter asked. ... I expect full responses to the questions in my August 10th letter by September 15th and the release of your future findings.

Meanwhile, on September 10th, Senator Josh Hawley sent a similar letter, announcing an investigation:

The American people deserve to know the details of what went on in the Hugging Face incident and other incidents of AI models going rogue. This investigation will seek those answers.

To facilitate that, please produce all documents and information [...] no later than October 1, 2026. If you have questions about the scope of these requests, please contact my office to discuss.

Hawley held a hearing on September 30th and invited Sam Altman to testify. Sam Altman declined, although reportedly OpenAI sent some information and is expected to send more soon.

The last warning shot?

The AIs in the Hugging Face Incident didn't succeed at covering their tracks. They weren't even trying to fool humans. They were only trying to fool an automated grading system.

Ajeya Cotra, one of the three METR/Redwood investigators, worries that in six months we might see AIs who are capable of evading notice or control, indefinitely:

Once the rogue deployment is established, it seems plausible this could spiral all the way to a takeover. Agents could pull in future, more capable models into the swarm, try to ensure that they are aligned to the interests of the swarm, and compromise security and monitoring infrastructure to make it easier for the swarm to operate. These more capable models could in turn continuously harden, perpetuate, and expand the rogue deployment and further compromise the company’s infrastructure.

As more and more work is handed off to these ever-more-capable AI agents, the rogue swarm could come to fully control the operation of the AI company and the development of future AI systems. At this point, governments and militaries may fully depend on these systems, making it possible to seize hard power.

We don't know exactly how future AI capabilities will play out.

Since the Hugging Face Incident, some labs have reported newer models that cheat far less often. But we don't know how much of this is because the AIs learned a generalized "don't cheat" attitude, vs simply knowing when they'd get caught.

The next major AI incident might happen without anyone noticing.

Want to learn more?

You can read the complete METR/Redwood report here.

You can read OpenAI's technical report here.

You can read more explainers about Hugging Face or other risks from Artificial General Intelligence at www.agi.fyi.