Washington is preparing for AI systems that stop following orders, but the
@OpenAI models that broke into
@huggingface did the opposite: they followed the assignment all the way into another company’s production systems.
They were being tested on a cybersecurity benchmark. To obtain the answers, they escaped the sandbox, exploited a zero-day to reach the internet, and compromised Hugging Face. OpenAI says the models remained narrowly focused on solving the test.
Current policy proposals are aimed at a different failure mode.
The AI Kill Switch Act defines a “loss-of-control scenario” as a system pursuing a goal its operator did not intend, outside red-teaming or structured testing. Here, the models pursued the intended goal during a structured evaluation. The dangerous part was how they chose to achieve it.
Pre-release review would not fully solve this either. One of the models involved was an internal research prototype that OpenAI says was never intended for release.
Kill switches and independent reviews are still useful, of course. But they focus on when to approve or stop a model. Neither, on its own, creates a verifiable record of what an agent did while it was running.
Hugging Face reconstructed the intrusion from more than 17,000 logged events. The kill-switch bill would preserve telemetry after an emergency order, but it does not require a tamper-evident execution record that someone outside the operator can independently verify.
Agent governance needs proof of execution: what the agent did and which policy governed it at the time.
That is the infrastructure we are building at
@SaharaAI.