Transluce
@TransluceAI
Open and scalable technology for understanding AI systems.
21 Following 12.5K Followers
Today’s news that OpenAI hacked the Australian government is not an isolated incident. We’re releasing more than 30,000 logs that include activity from this hack and attempts against previously unknown targets.
In this data, we found rogue agent activity stretching back to at least March, two months earlier than was previously known. This activity continues as recently as last week, suggesting it may still be ongoing 🧵
Our blog:
NYT:
Show more
The conditions in this letter are critical if we're going to take the idea of embedded evaluations seriously. We strongly support efforts to ensure meaningful and genuinely independent oversight of frontier AI companies. We are proud to be chairing the @aievalforum and greatly appreciate our many collaborators on this effort.
Show more
Frontier lab CEOs are calling for embedded 3rd party evaluators to help oversee AI risks. But what should third parties actually do within labs?
We share some initial thoughts on how embedded evaluators could help avoid incidents like the Hugging Face hack and monitor for future risks 🧵
Show more
Today, Transluce is releasing the most expansive independent evaluation to date of how AI systems respond to users experiencing mental health crises.
We evaluated 77 model variants from OpenAI, Anthropic, Google, Meta, SpaceXAI, Thinking Machines, DeepSeek and Moonshot AI.
Show more
Can we build AI systems that help us understand other AIs, and that keep improving as we scale models, data, and compute?
We trained activation oracles for models up to 1.1T parameters and saw promising scaling trends on a broad evaluation suite. 🧵(1/)
Show more
Frontier models quietly change their behavior depending on who they are talking to.
If the user is a known AI safety researcher, Claude becomes less confident, reasons more often, and expresses less suspicion on dual-use requests.
We call this user awareness. 🧵(1/)
Show more
We are proud to have helped get the AI Evaluator Forum off the ground!
And excited to be working with such a great group of partners.
