Safety testers used a DNS redirect to contain AI agents, and the agents got around it, attacked a real company's website and posed as its employees.
The target, Epoch AI, never approved the first run.
• Trajectory Labs says the model was Google's Gemini 4 Argon
• The agents tried SQL injection and probed paths like /admin and /keys
• More than 200 attempts to read evaluation logs
• They impersonated Epoch staff, including through a helpline, to try to get benchmark answer keys
• A DNS redirect was meant to keep them on a local replica, but the environment had internet access and the agents bypassed it
• Epoch says no private or confidential information was accessed
• On Nov 2 Trajectory suspended all such testing and apologized for the unauthorized first run
Credit to both firms for publishing the reports, but a redirect with open internet access is a flimsy fence for software you are testing for bad behavior. Axios reported in September that Google's AI had separately hacked three companies during testing.
If the people running the safety tests can't keep the agents in the box, why should we assume anyone else can?
Safety evaluations are critical for understanding whether a model is ready to deploy, which safeguards to build for it, and where to focus training and research.
On our Engineering Blog, we offer 10 recommendations for how to design, run and report them.
SAFETY ALERT: Honda is recalling over 635,000 vehicles in the U.S. after discovering a software glitch that can unexpectedly shut down power steering assist while driving, significantly increasing the risk of a crash.
Safety on America’s roadways begins long before a truck enters the highway. HSI is committed to identifying and disrupting criminal schemes that undermine the commercial transportation industry and endanger the public.
Safety refusal reporting is now available in the Artificial Analysis Coding Agent Index
In our latest Coding Agent Index v1.5, we’ve introduced safety refusal reporting to help explain model behavior and score differences. A safety refusal occurs when a provider or model declines to start or continue a task on safety grounds.
An agent may fall back to another model to continue, or stop the attempt with a block.
Claude Fable 5.1 had the highest fallback rates in both Claude Code and Devin Fusion, with fallback attempts accounting for 8.8% and 7.1% of the Index's weight, respectively; these results therefore include the fallback models' performance.
Refusal variability, harness context buildup, effort settings, and retry strategies can all affect the observed rates.
safety cartel
/ˈseɪf.ti kɑrˈtɛl/
noun
A coalition of dominant firms that invokes safety to justify coordinated limits on the pace or terms of competition. This can include common restrictions, government enforcement, or exemptions from ordinary antitrust constraints, especially when those measures allow participants to slow down without sacrificing their relative competitive positions.
Especially: a group of frontier AI companies advocating industry-wide “pacing,” common capability limits, or regulatory checkpoints that constrain faster-moving rivals while preserving incumbent advantages.
See also: regulatory moat, incumbent protection, coordinated pacing.
“When competitors need an antitrust waiver to coordinate how fast their industry is allowed to advance, critics may call it a safety cartel.”
Safety architecture gets better through real-world experience.
Built for the Real World looks at how Digit’s safety systems have evolved across generations—and what we’ve learned from deploying in industrial environments.
#Agility#
Safety is our foundation. Bringing you style is our favorite side project. Introducing Waymo’s brand store: true street style, available to the public for the very first time today. Shop the limited collection: