PatronusAI
@PatronusAI
Simulation research and infrastructure for human-aligned AGI
220 Following 2.6K Followers
Today, we’re releasing SpeedrunBench: the first benchmark that measures how fast agents can beat video games.
We asked frontier models not to simply complete video games, but to speedrun them - across 10 titles like Mario Kart and Pokemon.
On simpler games, agents get close to the human world record. However, on more complicated games like Pokemon Blue, the best models Kimi-K3 and Opus 5 are ~4x off the world record.
Benchmark, paper, and demo below.
Show more
Today we're excited to announce FigmaTrace, the first open source computer-use dataset for design.
We captured 200+ hours and 3400+ trajectories of real design work in Figma, recording step by step how designers execute complex, long-horizon tasks.
Dataset, paper, and blog below.
Show more
