It’s a bit perverse how AI-driven cyber attacks may create a huge demand for AI-assisted cyber defense. Not at all intentional I’m sure, but seems true all the same.
How useful is a new model, really? We can’t answer this for everyone’s work, but we can answer it for our own. And since we know our standards, we’re the right ones to grade these tasks. I’m very excited to track this through time and would love to see more orgs do this.
Can AI automate Epoch? We're introducing Epoch Automation Reports to evaluate frontier models on realistic, open-ended tasks drawn from our own work.
Claude Fable 5.1 and GPT-6 Astra lead, yet they are far from fully automating Epoch’s work.
@Nerland87 You might be right. Yesterday I was standing under an oak tree (of the type on the right there) with the 3yo, and he was picking up individual leaves and asking me “is this an oak leaf?” and this went on for 20 minutes. Like the cheese bit in Borat.
Incidentally, our results are less in tension with @pzeroresearch ‘s recent (very cool looking) research than it might seem! They found the same models to be superhuman at efficiently improving metrics that AI R&D often targets, but not…
Incidentally, our results are less in tension with @pzeroresearch ‘s recent (very cool looking) research than it might seem! They found the same models to be superhuman at efficiently improving metrics that AI R&D often targets, but not…