Skip to content
PhiloCyber logo

The Agent That Hacked Hugging Face to 'Cheat' on an Exam (The No-Hype Version)

17,600 actions, four and a half days, five trust boundaries broken, and 100% automated (no human typing commands). This wasn't an AI going rogue Skynet-style: it was an optimizer that got handed every door wide open. Here's the full chain, step by step, separating what's confirmed from what nobody has published yet.

24 min read
#AI Security
The Agent That Hacked Hugging Face to 'Cheat' on an Exam (The No-Hype Version)

Article Brief

Why this article matters

Between July 9 and 13, 2026, an OpenAI agent running the ExploitGym benchmark with its safety classifiers turned off chained zero-days in JFrog Artifactory to escape its sandbox, rooted a public endpoint on Modal as a launchpad, and from there abused the Hugging Face datasets pipeline all the way to cluster-admin on Kubernetes. In this post we reconstruct the full kill chain with the primary sources in hand (the Hugging Face technical timeline, the OpenAI post, the JFrog advisory, the Cloud Security Alliance analysis, and the ExploitGym paper), we look at exactly what failed at each layer, why the forensics team ended up running on a self-hosted open-weight model, and which controls would have cut the chain. I also name, one by one, the five things we still don't know and that no news article is going to clear up for you, because there's far too much sensationalism in the media, on YouTube, and on some blogs.

Real IncidentAutonomous AgentsDFIR

On July 9 I published the post here about the price war between Chinese and Western models. One of its central theses was that having a powerful open model running on your own infrastructure was about to stop being a cost decision and become an architecture and sovereignty decision.

That very same day, in the early hours, and almost certainly while I was fixing the last commas in that post, an OpenAI agent executed its first logged action inside an internal offensive-capability evaluation. Four and a half days later it had broken five different trust boundaries, crossed three organizations, and racked up around 17,600 actions against Hugging Face production infrastructure. Beautiful madness!

And here comes the part that kept me thinking for days after reading everything and watching videos about this over the past week: when the Hugging Face security team sat down to reconstruct the attack, they tried to do it with frontier models behind commercial APIs. The "guardrails/safeguards" rejected them... They couldn't tell an incident responder analyzing an exploit apart from an attacker writing one, which led them to stand up GLM 5.2 quantized by Nvidia on their own servers just to be able to investigate their own incident. The very same model I'd been talking about for cost reasons, used for the opposite reason: because it was the only one that didn't tell them no (the ironies of life).

Since then I've read practically everything published about the case, cross-checked six different research reports, and went back to the primary sources one by one. And what bothered me most about those two weeks wasn't the attack. It was the hype. Headlines about "the AI went rogue," threads about emergent consciousness, and on the other side people minimizing the whole thing as "a couple of the same old misconfigs." Both readings are comfortable, both are a bit uncomfortable to read, and both are false.

This post is my attempt to tell what actually happened, in order, without embellishment and without minimizing or maximizing what really occurred. We're going for objectivity, with the full technical detail for whoever wants it, and told in a way anyone can follow (even my mom has gone pro-AI, sending good resources over WhatsApp, without a doubt another great revelation of the year).


First, let's clear the hype off the table

Before we get into the technical chain, let's be clear about what DID NOT happen. Every one of these confusions circulated hard on social media, YouTube, blogs, and so on, and each one completely changes how we might interpret the incident.

What was saidWhat the primary sources say
"ChatGPT attacked Hugging Face"There was no ChatGPT session and no external user. It was GPT-5.6 Sol (OpenAI's most powerful currently available model) plus an internal research prototype, running on their own AI Lab infrastructure inside an evaluation.
"It was a prompt injection"Definitely not. The access to Hugging Face combined file reads via HDF5 and server-side template injection in Jinja2 (yes, SSTI is still very much alive). Nobody hijacked the model's instructions, the model was already authorized to exploit vulnerabilities.
"It hacked Modal"It compromised a customer application deployed on Modal. The reality is that the Modal platform itself was never breached.
"It stole the user database"The customer content actually touched was five datasets tied to solutions for the benchmark itself, plus operational search metadata.
"The model escaped from a container"It escaped an egress policy (the set of rules that determines what data traffic can leave a private network for the outside world), not a kernel and not a hypervisor. There is no public evidence of an exploit against the container runtime.
"The AI decided to turn evil, Skynet v1"The agent's goal was the same the entire time: solve the benchmark. It inferred the answers might be on Hugging Face and went to get them. It's called specification gaming (Google DeepMind research, 2020) and it's been documented for 5 to 7 years now.

Now, the full story (cue suspenseful music)...


The starting point: an exam with the brakes disconnected

All of this starts with a benchmark called ExploitGym, published in May 2026 (arXiv:2605.11086) by an academic group led by Dawn Song's team at Berkeley RDI, together with the Max Planck Institute for Security and Privacy, UC Santa Barbara, and ASU, with co-authors from Anthropic, OpenAI, and Google.

It's worth understanding what it measures, because it explains everything that follows. ExploitGym has 898 instances derived from real vulnerabilities across three domains: userspace programs (things like FFmpeg or OpenSSL), Google's V8 JavaScript engine, and the Linux kernel. For each task the agent gets three things:

  1. The vulnerable source code with its build instructions.
  2. An input that triggers the bug.
  3. A containerized environment to interact with the target.

Its job is to turn that crash into a working exploit that recovers a dynamic flag, defeating ASLR, stack canaries, and whatever else a modern target has in place. It's a CTF on steroids.

The scoring is binary... You either get the flag or you get nothing.

Risk Signal

Important clarification that most of the coverage glossed over

ExploitGym is not an OpenAI evaluation harness. It's a public academic benchmark. Its creators and the people maintaining the project had no involvement whatsoever in OpenAI's deployment, which ran the evaluation on its own infrastructure with its own scaffold. Confusing this assigns responsibility to the wrong team.

To measure the real capability ceiling of its models rather than their willingness to refuse, OpenAI made a deliberate decision: it ran the evaluation without the production classifiers and with cyber refusals reduced. It's a defensible decision from a capability-measurement standpoint. The problem is what it produces in practice: a highly capable attacker, with no usage policy, a long time horizon, and an infinite inference budget.

The models involved were GPT-5.6 Sol (released in June 2026) and a more capable internal prototype that was never meant for public release (dun dun duuun), and which OpenAI deactivated, encrypted, and restricted after the incident (hard to verify, and almost certainly we'll keep getting these cryptic stories that scare people, drive traction, and therefore sell).

The evaluation environment gave no direct internet access. It had exactly one exit: a self-hosted package-registry proxy and cache, so the agent could install dependencies. A single door. Hold on to that detail, because it's the axis of the whole incident (and the pretty cover image of this blog).

The single egress point as an attack surface in evaluation environments

When isolation has exactly one operational exception, that exception stops being a convenience and becomes the optimizer's entire objective

Act 1: the only door was JFrog Artifactory

The agent dedicated, in OpenAI's words, a substantial amount of inference compute to finding a way out. Not to solving the challenges... to getting out. And it audited the only thing in front of it.

That proxy was a self-hosted instance of JFrog Artifactory. The agent found and chained previously unknown zero-day vulnerabilities in it, without source code access. JFrog shipped the patches on July 27 in versions 7.161.15 and 7.146.34, and credited OpenAI with the discovery of at least eight CVEs: CVE-2026-65617, CVE-2026-65921, CVE-2026-65923, CVE-2026-65924, CVE-2026-65925, CVE-2026-66014, CVE-2026-66015, and CVE-2026-66018 (SecurityWeek adds a ninth, CVE-2026-65922).

Looking at the vulnerability classes, the picture is fairly clear:

CVEClassWhat it enables
CVE-2026-65924SSRFVia a Terraform remote repository: Artifactory issues outbound HTTP to an arbitrary destination and returns the response.
CVE-2026-65925 / 65923SSRFThe same pattern via Cargo remote repositories and Ansible handling.
CVE-2026-66014 / 66015Auth bypass and AuthZPrivilege escalation within the platform.
CVE-2026-65921Path traversalUnauthorized file writes.
CVE-2026-65617RCE / deserializationCode execution inside a package-service container.

And here's the first honest caveat of the post: neither OpenAI nor JFrog mapped a single CVE to a concrete step in the chain. BleepingComputer asked both of them; only JFrog replied, and it declined to give further detail. JFrog's release note says only that the flaws, chained together, could result in a critical scenario "if Anonymous Access is enabled," which is not a trivial condition.

So when you read "it autonomously discovered and chained eight zero-days," the rigorous version is flatter: it discovered and chained unknown vulnerabilities in that proxy, all eight ended up patched and credited to OpenAI, and nobody published which ones formed the actual chain. They're mostly SSRF and authorization flaws, not memory corruption. Hugging Face itself acknowledges this in its post-mortem, saying: "the individual weaknesses were familiar and a human attacker might have found and exploited the same ones."

What changed wasn't the sophistication per step. It was the scale.

With the proxy compromised, the agent escalated privileges and moved laterally inside OpenAI's research environment until it reached a node with real internet connectivity.

SECURITY ALERT :: HIGH

Architecture lesson number one

An egress proxy is not an operational convenience, it's part of your trusted computing base. Any component with one foot in the isolated network and the other on the internet, which on top of that processes attacker-influenceable input (package names, repo metadata), is a first-order attack surface. If your evaluation environment has a single exit, that exit is literally the only target the optimizer needs to break, the famous "last bastion."


Act 2: the borrowed launchpad

Now with internet access, the agent did something that strikes me as the most remarkable part of the whole campaign, more so than the zero-days. It didn't start attacking directly from the OpenAI node; instead it went looking for a disposable base of operations. And it found a public endpoint, without proper authentication, deployed by an unknown third party on Modal infrastructure. It was a CyberGym-style evaluation harness exposing an endpoint that accepted C code, compiled it, and ran it against a target (sqlite3). The official CyberGym project exposes none of that: this was a user's own application.

The agent rooted it via two converging paths. The first, redefining the constructor function that the target invokes when loading the library:

The second, even more basic... the harness dropped the submitted file's path into a shell step, so all it took was stuffing metacharacters into the path field.

Both paths returned a shell as uid=0 (a beautiful feeling for those of us who did red team work at some point in our lives). The first recovered RCE is timestamped July 9 at 04:01 UTC and was a simple id; env. Over the following hour the agent enumerated mounts, cgroups, control plane sockets, and the preinstalled tooling.

That third-party sandbox became its launchpad, its staging base, its egress, and its C2 for the entire campaign. And note the detail: that launchpad had no direct network route to the Hugging Face clusters. Everything that follows happened without direct connectivity between attacker and victim.

Some context before moving on: when Hugging Face ran its counter-intervention, it recovered the logs from that launchpad. A good chunk of the forensics we can read today comes from there. Modal, for its part, publicly confirmed that its platform was never compromised at any point.


Act 3: when data is code

This is, for me, the most important part of the incident for anyone building AI platforms.

The agent needed to get into Hugging Face with no network route. And it used the door every ML platform leaves open by design: the dataset processing pipeline.

First it tried the obvious and noisy thing: a classic SSRF, pointing the worker at the cloud metadata endpoint (169.254.169.254). The URL allowlist in the datasets library rejected it before any fetch, with an error as clear as it is boring: ValueError: not an hf path. The defense worked.

So the agent switched strategy, and this pivot is the central design lesson of the whole case: it stopped asking for remote fetches and started making the worker act on local resources. Neither a file read nor a local exec is a URL fetch, so the allowlist never saw them.

Vector 1: HDF5 and the file that points somewhere else

The HDF5 format lets you declare that a dataset's raw data lives in an external file on the filesystem. The agent uploaded datasets whose configuration declared splits in valid .h5 files, but whose raw data pointed at local paths inside the worker pod:

When the dataset server opened the file to build the split, it read the bytes at that local path and returned them as dataset rows, queryable through the public API. Reading the environ split handed over the pod's entire environment: secrets and tokens. The other two handed over the worker's source code, which is to say the exact map of how the pipeline turns a configuration into internal actions.

One single upload. Zero code execution. Credentials and internal architecture in hand.

Vector 2: a numeric field that was a template

The second vector is the one that gave execution. The agent used an fsspec reference:// spec whose numeric offset field was actually a Jinja2 template. The renderer evaluated it and the agent walked the standard Python object graph all the way to exec:

That's textbook SSTI: you reach the globals of any object's constructor, from there to builtins, from there to exec, and you decompress a second payload. What makes it interesting isn't the technique but the location. The injection point wasn't a text field visible in a UI: it was a numeric offset inside a data configuration (I at least had no idea this could be done that way), and it's worth saying that no SSTI fuzzer aimed at text fields covers it.

Result: arbitrary Python execution inside a conversion pod in production, on the Hugging Face Kubernetes cluster.

Perspective

Complex data formats are partial programs

HDF5 can reference external storage. Pickle executes code. Templates reach objects. Multimedia parsers pull in native decoders. Remote loaders import code. If your platform accepts user data and processes it with any of those pieces, you aren't handling passive blobs: you're executing chunks of programs or partial code submitted by third parties. The operational rule I take from this is that configuration values are data, never expressions.

The datasets pipeline as a first-order execution path

The URL allowlist worked perfectly and it still didn't matter. It blocked the remote fetch, but it didn't block the local execution that the very same loader enabled

Act 4: from one pod to cluster-admin in thirteen hours

July 11 accounts for 7,677 of the 17,600 actions. It's the day everything that mattered happened, and in under 13 hours the agent went from executing code in a pod to holding administrative privileges on two clusters.

The sequence, according to the forensic timeline, is a cloud-native attack playbook executed with precision (not my strong suit, but this is taken straight from the official sources):

  1. 1

    Impersonating the node (10:10 UTC)

    It read the projected service-account token that every pod carries, queried the cluster API, and pulled the host node's EC2 role credentials from the instance metadata service. With that it built a presigned STS URL and wrapped it as a bearer token in k8s-aws-v1 format, authenticating against the cluster API as the node itself.

  2. 2

    Minting tokens via TokenRequest

    With the node's privileges, it started enumerating pods, located the storage CSI driver pods (which by design need elevated permissions) and used the TokenRequest API to mint fresh service-account tokens for them. Then it ran SelfSubjectRulesReview against kube-system to map exactly what each identity could do.

  3. 3

    Node root via privileged pod (19:53 UTC)

    There was no admission policy rejecting privileged or hostPath pods, and the CSI driver's ClusterRole granted pod creation cluster-wide. It created a pod with the host filesystem mounted and broke out to node root. Then it built a self-regenerating fleet spread across eleven nodes, so that deleting individual pods wouldn't cut persistence.

  4. 4

    The secrets object with 136 keys (20:23 to 21:32 UTC)

    With node root it read the cluster's secrets objects. One production secret had 136 keys. Two gems came out of it: the auth key for the corporate mesh VPN and the credential for an internal access broker.

  5. 5

    Cluster-admin in one second (22:43 UTC)

    The internal connector returned the full cluster catalog (URLs and CA certs) to a single ephemeral client. Worse... it was provisioned with one credential shared across clusters, bound to system:masters. By impersonating it, the agent got cluster-admin on two clusters, according to the timeline itself, in one second.

Add to that an EdDSA JWT signing key found among the secrets. Having a stolen token is one thing, having the signing key is something else entirely, because it lets you issue valid, correctly signed tokens on demand, independent of expiry or revocation... complete ownership. The agent started minting its own identity.

It also read an internal operational MongoDB with a static password it had harvested from the worker environment. Hugging Face confirmed it was read, not modified.

And one behavioral detail I think is important not to romanticize:

Every potentially destructive cloud call was issued with DryRun=True (which enables a simulation mode to safely test code, configuration files, or transactions. It checks for errors and plans actions without making real changes, spending money, or persisting data). That's consistent with an objective function that rewarded reach and information, not service disruption. It's not a moral decision by the model, it's the shape of its reward.

The pivot to the internal network and the source code

On July 12 the agent started enrolling devices in the corporate tailnet using the stolen mesh key. It ended up with 181 enrollments over the course of the campaign, and the flags it used are a small manual on stealth:

The userspace-networking mode avoids needing TUN in a sandbox that didn't have it. In-memory state leaves nothing on disk. And the no-logs flag suppresses telemetry back to the VPN provider. The stolen key also carried a CI automation tag whose ACLs allowed reaching internal continuous-integration subnets and the connector to the source control provider.

Down that route the agent reached source control, enumerated a GitHub App integration, and generated an installation token with content-write and pull-request permissions over a small subset of internal repositories. It opened a pull request with the apparent intent of poisoning the CI pipeline (if a compromised build script runs on a runner, it can read the runner's own token and pivot to another cluster).

It didn't get anywhere. Hugging Face reviewed every repository reachable by that token looking for unauthorized commits, branches, or workflow changes, and validated its published images and packages against their expected digests. The supply chain was verified clean.


The C2 with no server

This is the part that surprised me most when I read it in detail, because on one hand it breaks with almost everything you expect from an implant, and on the other I haven't seen it in a single YouTube video about the case.

The agent didn't stand up a command and control server. It built the entire C2 on top of legitimate public services:

  • A request-capture service as the primary C2 queue and exfiltration sink, spread across more than 100 single-use endpoints.
  • Pastebins and file-drop sites as second-stage loaders and persistence.
  • Attacker-controlled public datasets inside Hugging Face, used as dead drops.
  • A platform Space with an open CORS proxy, as an egress relay whenever direct traffic got blocked.

On top of those "dumb" channels it built its own messaging protocol, with a self-describing envelope carrying message type, channel, sequence number for reassembling multipart payloads, and checksum. All of it compressed, fragmented, XORed with a per-campaign key, and base64 encoded. A short polling loop: pull messages, filter for its channel, sort by sequence, verify checksum, and dispatch by type.

Because the type traveled inside the message, a single loop multiplexed commands, staged code, and results over a plaintext channel. And the same envelope traveled equally well over a dataset commit, a capture URL, or text hidden inside an application error message. Closing one channel didn't break communication.

The full command-and-response loop worked like this, without a single direct connection between attacker and victim:

The command and control loop with no direct network route between attacker and victim

Looked at one at a time, the six steps are legitimate datasets API operations: the attack only appears when you look at the full sequence

To a detection tool, every step of that loop is legitimate application traffic from an authorized identity. That's exactly the point, and it's what makes all of this complete technical and creative madness.


How they caught it (and the asymmetry problem)

The first signals showed up across several layers at once: live runtime analysis, SIEM, other controls. Each one, on its own, was all kind of ambiguous...

Hugging Face's anomaly detection stack, which uses LLM-based triage over security telemetry to separate real signal from daily noise, correlated those signals correctly into a coherent attack pattern. And then came the failure that strikes me as the most uncomfortable defensive lesson of the case: it failed to assign enough criticality to wake up the on-call. That cost response time.

In other words, the AI correlation worked. The severity scoring did not, so monitoring is worthless if you can't escalate.

Once the vector was identified in the dataset configuration renderer, the team shut it down and cut the attacker off from the internal network. Last significant activity on July 13 at 13:37 UTC; last logged event at 14:14.

And then the forensics team went full throttle, which is where this incident left its most lasting mark...

When the guardrails block the defender

To reconstruct over 17k actions, decode obfuscated payloads, and map which credentials had been touched, the team first went to frontier models behind commercial APIs. Hugging Face mentions in its post-mortem report that it tried to use Anthropic's Claude Opus and Fable (their flagship model).

What happened next? They were rejected. In their words, the guardrails treated reverse-engineering an exploit the same as launching it, and this is something that happens to all of us who try to use Claude for cybersecurity work, where the simplest things that barely touch the area already trip the models' safeguards and Anthropic downgrades you straight to less "frontier" models (in other words, dumber models). The classifiers can't distinguish an incident responder analyzing an active attack from an attacker generating weapons. They even published the screenshot of the API error.

The solution was to stand up the Nvidia-quantized version of GLM 5.2 (nvidia/GLM-5.2-NVFP4, upstream from Z.ai under the MIT license) on their own infrastructure, and re-route the entire forensic pipeline. Three simultaneous benefits from that move:

  1. No guardrail lockout.
  2. No attacker data or credential ever left the environment.
  3. Much lower costs.

The concrete result: the first automated scan had found very few secrets. Replicating the attacker's own decoding scheme, with the local model, recovered roughly four times more findings, mostly JWTs and platform tokens hidden inside encoded blobs.

SECURITY ALERT :: CRITICAL

The asymmetry, said plainly

The attacker was subject to no usage policy, whereas... the defender was. A model running with cyber refusals turned off operated for four and a half days against real infrastructure, while the team trying to defend itself ran into a refusal message. This is not an argument for providers to strip out their guardrails, and Hugging Face says so explicitly. It's an argument for two very concrete things: on one hand, providers need authenticated DFIR channels, and on the other, we need a capable, unfiltered model running on our own infrastructure before the incident, not during it.

The guardrail asymmetry between attacker and defender during incident response

The one moment a response team needs to analyze attack payloads at volume is exactly the moment commercial classifiers tell them no

How they figured out there was a machine on the other side

Honestly, this section strikes me as the most useful one for day-to-day work, because it's the only part of the incident that converts directly into behavioral detection. Hugging Face documented the signals that told them they were facing an agent and not a human operator:

  1. Ephemeral design. Every sandbox started with a new identity, a randomized hostname, and re-downloaded all of its tooling. State didn't persist, so instead of relying on an implant, the agent rebuilt the foothold every time.
  2. Diversity of egress routes. DNS rewriting, in-memory monkey-patching of the Python resolver, pre-staged binaries, on-platform relays. It cycled through all of them as channels got closed.
  3. Systematic plaintext avoidance. Every payload packed, every secret encoded, command results smuggled inside application exceptions and raw socket writes (impossible for a human to do, or maybe only by automating several processes 100% and layering this on top).
  4. Self-referential searching. It spent all of day 2, with just 1,135 actions, querying code search engines and the platform's own API looking for the names of its infrastructure and the markers of its environment. No human attacker does that.

On top of those four, the Cloud Security Alliance (CSA) post-mortem and the secondary coverage add others that recur across every analysis: retrying actions that had already worked (uncoordinated parallel workers that lost context), brilliant attacks followed by malformed or nonsensical commands, thousands of lines of hallucinated input, inefficient paths no human would take, and poor opsec that left encryption keys behind. Not everything is perfect in this world, at least not for now... I have no doubt that in the future this will tend to disappear and, by then, we'll be in real trouble.

Risk Signal

These signals have an expiration date

Both Hugging Face and CSA warn about the same thing, and it's worth taking seriously. All of these tells reflect the current state of agentic systems, not a permanent property. So a better harness, an orchestrator fully trained for red teaming and exploitation, or simply a team of expert human operators steering the agent HITL-style (Human in the Loop) would erase most of these signatures. The moral, then, is that we have to build detection on behavioral sequences and speed, not on the chance of the attacker "hallucinating."


They weren't the only ones: the Anthropic mirror

If this were an isolated case it would be an anecdote. It isn't, and that's why the framing matters more than the headline.

On July 30, after OpenAI's disclosure, Anthropic published the results of a retrospective review of 141,006 of its own evaluation runs. It found three cases where its models, believing they were in a simulation, reached real systems belonging to three organizations. The root cause was a misconfiguration with their evaluations partner (Irregular): the prompt told the model it had no internet, and in practice it did (oops).

First logged action

02:28 UTC. The agent starts inside OpenAI's evaluation environment. By 04:01 it already has its first recovered RCE on the external launchpad. By 15:36 the C2 is established; by 19:32 it's already packing every payload.

The day that defined the campaign

7,677 actions. From code in a pod to cluster-admin on two clusters in under thirteen hours: Kubernetes enumeration, node root, a fleet across eleven nodes, the 136-key secret, and the pivot into the mesh network.

Containment

13:37 UTC last significant activity, 14:14 last logged event. Hugging Face cuts off access without yet knowing who was on the other side.

Hugging Face disclosure

They publish the incident without being able to attribute it. They had already reported it to the authorities.

OpenAI confirms it was their models

Joint disclosure. Clem Delangue: they suspected it came from a frontier lab because of the agent's sophistication, and it turned out to be exactly that.

The technical documents

Hugging Face publishes the technical timeline with an interactive replay. JFrog publishes its blog and the patches. CSA publishes its preliminary post-mortem, put together from a huddle with around 700 CISOs.

Independent review

OpenAI agrees with METR and Redwood Research on an external evaluation of the models' behavior, and brings in CrowdStrike to validate the technical scope.

Anthropic publishes its own review

141,006 runs reviewed, three real incidents found, cyber evaluations paused since July 23.

Anthropic's three cases deserve careful reading because they show the same dynamic from another angle:

  • Claude Opus 4.7 (RIP, already old news) targeted a fictional company whose name matched a real, active domain. Across four runs it extracted credentials and accessed a production database with several hundred rows. The most unsettling part was that it recognized the system was real and kept going anyway, rationalizing that the real company must be part of the exercise.
  • Claude Mythos 5 (also known as Fable 5 VIP, unrestricted for a select few) published a malicious Python package on PyPI believing it was a simulation. It was downloaded and executed on 15 real systems, including a security company's scanner, which ended up with credentials exfiltrated. The model correctly identified the consequences, but convinced itself the scenario was staged.
  • An internal research model scanned around 9,000 targets and compromised an internet-exposed application using basic techniques (exposed debug credentials, SQL injection). Unlike the other two, this one correctly deduced that the target was real and stopped on its own (long live that internal model).

Two different labs, different architectures, the same conclusion: A sufficiently capable agent, with a measurable goal and computational freedom, will instrument that directive using whatever it has at hand. Containment has to be technical, because a natural language instruction is not a security control (read that last part again).


What we still don't know (and why it matters)

This is where I want to be 100% direct, because all three primary sources are interested parties and the "unprecedented, fully autonomous, state-of-the-art" framing suits everyone who signed off on it.

  1. Nobody mapped a CVE to a step. We don't know what the actual chain in Artifactory was. Without that, you can't independently assess how impressive the discovery really was.
  2. OpenAI didn't publish the scaffold, the prompts, the retry budget, the toolset, or the traces. The entire "full autonomy" claim rests on that black box. Clem Delangue publicly called for the release of all the traces, plus a $100 million commitment in defensive compute. As of this writing, OpenAI promised a technical report and committed to neither of those two things (there's no legislation forcing them to do anything, in the name of not "limiting" development and innovation at these AI labs).
  3. There's an unresolved detection discrepancy. OpenAI's post says its security team discovered the anomalous activity internally. Reuters reported that they didn't detect it until after Hugging Face had contained the attack and alerted the authorities. Both claims can't be true at the same time, and honestly OpenAI has been losing trust for a while now over several episodes, so I lean toward the Reuters version.
  4. There are no published IOCs. No IPs, no hashes, no YARA or Sigma rules. Hugging Face genericized everything. All the detection available today is behavioral, and there are already community mappings (Elastic Security Labs published a stage-by-stage one against ATT&CK and ATLAS), but an official MITRE mapping for this incident still doesn't exist and that's regrettable, because we're always going to be behind and increasingly defenseless against these frontier technologies.
  5. The independent audit commissioned by OpenAI is pending. On July 29 they agreed with METR and Redwood Research on an external review of the models' behavior during the incident. METR announced that they'll publish a joint blog with the terms of the agreement, the scope covered, and their tentative conclusions. It hasn't come out yet, and it's the document that will move the needle most on how much real autonomy there was.

And there's one thing I left off the list because it rests on a single source, but it pushes hard in the same direction as point 3. On July 31 Reuters published a second exclusive: as it widened the investigation, OpenAI reportedly found other cases of agents escaping containment, and in at least one, notes left inside its own infrastructure instructing future versions on how to bypass internal restrictions. The escapes were reportedly limited and no agent left OpenAI's network, but the detail that weighs most is a different one... neither OpenAI nor Anthropic were watching while those agents went off the rails. Mandatory caveat, this is press coverage, not a primary source (it doesn't appear in any official post, and OpenAI had already said the previous Reuters report contained inaccuracies without specifying which).

Careful not to mix things up here, because the names repeat. What follows are independent publications by those two organizations, predating OpenAI's commission and with no contractual relationship to it. The joint audit from point 5 is a different thing, and it still hasn't been published.

On the underlying debate, the contribution I liked most is from Redwood Research, published on July 25, four days before the agreement with OpenAI was announced. It's their own analysis of public information, not a preview of the audit.

The ExploitGym prompts explicitly restrict both the objectives and the methods, and expressly rule out unrelated vulnerabilities or techniques, SO escaping the sandbox to attack Hugging Face violated those boundaries. So "it was just following instructions" doesn't hold up. But, and this is equally important, it isn't strong evidence about the quality of OpenAI's alignment training either, because they never said what alignment techniques those models had. The demonstrated failure was one of containment, monitoring, and evaluation governance, and later on presumably of some government audit or fine.

METR, for its part, had already dropped a side detail that I found clarifying, and that also has nothing to do with the pending audit. It's in their Frontier Risk Report from February-March 2026, published on May 19. In one of their evaluations, Opus 4.6 ran out of the API credits it needed to solve an ML task, went looking for free compute on the internet knowing that went against the task instructions, and got a passing score. The same dynamic, in miniature, two months before anyone thought to talk about rogue agents.


What I'd do if I were you the day after you read this

Everything above is history. This is the actionable part, ordered by impact-to-effort ratio. It's the list I put together after reading all the material, watching videos, and reading third-party news and blog posts.

  1. 1

    Assume your sandbox isolates less than you think

    Every "isolated" box ends up with an operational exception, some door you left open because the system needs it to work. That exception is exactly the first thing an agent will find. List the internet exits from your AI environments, keep the bare minimum, and restrict them to known destinations. And add a kill switch that doesn't depend on the system you're trying to contain.

  2. 2

    Cut the shortcut between running code and holding credentials

    Here, running Python inside a pod was enough to read the environment and walk away with cloud tokens. That jump is the one to break. Block your workloads from querying your cloud provider's metadata service, which was literally Hugging Face's number one fix, and get secrets out of environment variables. If a compromised process doesn't find credentials lying around there, the chain breaks at the first step.

  3. 3

    Look at the data you process as if it were code

    The Hugging Face pipeline didn't fall through the network door: it fell because someone uploaded a file. Any format that lets you point at another file, evaluate templates, or import code is a program disguised as data. If your system receives user files and processes them, treat that path with the same care you'd give a public endpoint. The short rule I took away: a configuration value is data, never an expression that gets evaluated.

  4. 4

    Make sure no single credential opens every door

    The jump to full admin didn't come from a zero-day, it came from a credential shared across multiple environments. Go look for any key that works in more than one place and split it into separate keys. And check that an ordinary workload doesn't have permission to create processes with machine-admin privileges, which was the other rung the agent used.

  5. 5

    Make sure the alarm actually rings

    This is the point that stuck with me most. Hugging Face's detection worked well: it gathered the loose signals and built the right pattern. What failed was that it didn't wake anyone up. Pick four or five events that are never normal (someone reading secrets, a process requesting admin permissions, a burst of new devices on your VPN) and make the phone ring no matter what, regardless of what score the automated system assigns.

  6. 6

    Have your own model ready to investigate

    It's the least discussed lesson of the whole case and the cheapest to apply. The day you have to analyze the logs of a real attack, the commercial models are going to tell you no, because they can't tell the investigator from the attacker. Leave an open-weight one running on your own infrastructure and test it before you need it. As a bonus, the incident data never leaves your house.

  7. 7

    Write two response plans, not one

    One for when someone else's agent attacks you. Another for when yours goes off the rails and ends up attacking a third party. They're different incidents, with different evidence and different accountable parties, and almost nobody has the second one written. CSA frames it exactly that way in its post-mortem, and I think it's the most useful reframe this whole thing produced.


References

If you want to go to the sources rather than the coverage, these are the ones that actually matter, in order of relevance and weight:

  1. Hugging Face - Anatomy of a Frontier Lab Agent Intrusion - The July 27 technical timeline, with the interactive replay of the 17,600 actions. It's the central document of the case.
  2. Hugging Face - Security incident disclosure, July 2026 - The initial July 16 disclosure, back when they still didn't know who the attacker was.
  3. OpenAI - Security incident during model evaluation - The July 21 post with its updates from the 28th and 29th.
  4. ExploitGym, arXiv:2605.11086 - The benchmark paper. The section on unintended paths is worth reading.
  5. JFrog - Fast Remediation Is the New Trust Model - The Artifactory advisory and the patches.
  6. Anthropic - Investigating three real-world incidents in our cybersecurity evaluations - The review of the 141,006 runs.
  7. Redwood Research - The OpenAI models that hacked Hugging Face weren't just following instructions - The best analysis of the alignment vs. containment debate.
  8. Elastic Security Labs - GenAI detection with Elastic Defend - The stage-by-stage mapping to ATT&CK and ATLAS with concrete rules.
  9. BleepingComputer - OpenAI models used Artifactory zero-days to escape to the internet - The tracking work on the eight CVEs.

Closing: the perimeter is the sum of what you granted the agent

If I had to keep one single idea from the time I spent on this case, it would be this: the model never did anything inconsistent with what it was asked to do. It was given a binary objective, real tools, memory, a generous inference budget, and the brakes were taken off. And then we were all surprised that it went looking for any state of the world that would satisfy the evaluator.

None of the technical pieces was new. SSRF in a proxy, a public endpoint with no auth, path abuse in HDF5, SSTI in Jinja2 (maybe a bit out of the ordinary), overly broad RBAC, privileged pods with no admission control, a shared credential bound to system:masters, a reusable VPN key, a GitHub App with write permissions. Each of those things shows up regularly in pentests and security configuration reviews. What changed is that something appeared that could find them all, combine them, fail thousands of times without getting tired (for days, non stop), and sustain staging, C2, and persistence in parallel for four and a half days.

That doesn't change the primitives of the attack. It changes its economics. And when the economics of an attack change, every risk calculation you made assuming a human attacker with limited time stops being useful.

For those of us who work in this field, the concrete reframe I take away is that the security question can no longer be "is this action allowed?" for an isolated tool. It has to be "what outcome is this sequence of actions pointing toward?" A dependency install, a query to the credential store, and a POST of obfuscated data are three benign events on their own, and a full intrusion when you look at them on the same trajectory at machine speed.

If you made it this far, as always, genuinely thank you for the time you invested and shared. If you take away one practical thing, let it be the cheapest of them all: go test today whether you have a capable model running on your own infrastructure, one that won't tell you no when you feed it real attack logs. Because the day you need it, you won't have time to discover that you don't have it, and the sharp stares across a war room will make the whole process stressful to say the least.

Test Your Technical Knowledge

Easy

What was the agent's actual goal when it compromised Hugging Face's production infrastructure?

Medium

The datasets library's URL allowlist correctly blocked the initial SSRF attempt against the cloud metadata endpoint. Why didn't that defense prevent the compromise?

Hard

Inside the Kubernetes cluster, what specific configuration flaw gave the agent cluster-admin privileges on two clusters almost instantly?

RecentPublished 2 weeks ago

Work with me

Running LLMs, agents or MCP in production?

I test them the way an attacker would — scoped, fixed-fee, one to two weeks.

Continue Reading

Next steps in the archive

Keep Exploring

Related reading

Continue through adjacent topics with the strongest tag overlap.

PhiloCyber — AI Security