Apollo Research
@ApolloResearch
Our goal is to secure frontier AI systems from development, to deployment and governance.
0 Following 13.1K Followers
We have never seen a model do this before, but GLM-5.3 sometimes reasons (unprompted) about how its current actions could become training data and impact its future self. 🧵
To develop frontier AI safely, developers must be able to show that their models are not scheming, i.e. covertly working against them in pursuit of unintended goals.
New post: four claims any scheming safety case must make, and the resources embedded evaluators need to verify them 🧵
Show more
Frontier AI companies have committed to giving outside evaluators employee-like access to training, evaluation and deployment. We're very excited about this. But its impact depends heavily on implementation.
Today we're sharing our principles for embedded evaluations. 🧵
Show more
New Post: External testing needs embedded evaluators with employee-equivalent access.
Recent incidents mostly occurred during model development and internal evaluations, while current third-party evaluations happen before public release
Embedded evaluations can close that gap
Show more
We’re excited to see Dario and Sam commit to “embedded evaluators who have employee-like access to verify safety practices and report incidents.” [...] “whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.”
We’ve long advocated for third-party evaluators to have deep access. Risks from internal deployment and scheming cannot be appropriately assessed without direct insight into the training pipelines and internal deployment practices.
Apollo is looking forward to working on Embedded Evaluations with all Frontier AI developers. 1/3
Show more
As always, Apollo is hiring! New this time around: Founding Staff for our DC office.
It's becoming increasingly urgent and impactful to accurately communicate cutting edge research and its implications to the USG, orient their actions, and advance meaningful interventions. Help us with this impactful work and apply here:
Show more
We’re doubling down on our control and monitoring research. We have recently successfully red-teamed Anthropic’s auto-mode and will run more such campaigns in the future with multiple labs.
We will also continue to make better monitors ourselves and improve the research frontier for the blue team.
There are many low hanging fruits to pick for monitoring and not a lot of time. Our main blocker is having more hands on deck.
Come join us!
RS (Control):
AI Red Team Engineer:
AI Security & Control Researcher:
Every role is open in both offices, London and San Francisco. We provide visa sponsorship.
Our monitoring agenda:
Auto-mode red-teaming:
All open roles:
Show more
Apollo Research ran its first external red-teaming campaign for Anthropic’s auto mode.
Now auto mode is the default permissions mode in Claude Code. After hardening based on Apollo’s findings, the classifier's miss rate fell from 12% to 7%. 🧵
Show more
Our policy blog is a collaboration between @apolloresearch and @AVERIorg. It draws from an unpublished multi-author working paper led by @AVERIorg as well as published system card and internal research conducted by Apollo Research’s Science team and an Apollo Research project co-run with OpenAI. Full blog here:
Show more
Black-box access may soon no longer be enough to robustly make or verify safety and security claims. Deeper, white-box access is a necessary update to counter 'evaluation awareness' and keep loss-of-control evaluations state of the art. A new policy blog explains why. 🧵
Show more
