Anand Kannappan(@anandnk24):Today, we’re releasing SpeedrunBench: the first benchmark that measures how fast agents can beat video games. We asked frontier models not to simply complete video games, but to speedrun them - across 10 titles like Mario Kart and Pokemon. On simpler games, agents get close to the human world record. However, on more complicated games like Pokemon Blue, the best models Kimi-K3 and Opus 5 are ~4x off the world record. Benchmark, paper, and demo below. @PatronusAI

Anand Kannappan

@anandnk24

co-founder and ceo @PatronusAI (series B). prev AI @Meta evals, RL, data ⚡️

加入 May 2012

573 正在關注 1.6K 粉絲

Anand Kannappan@anandnk24

2026.09.03 19:20

Today, we’re releasing SpeedrunBench: the first benchmark that measures how fast agents can beat video games. We asked frontier models not to simply complete video games, but to speedrun them - across 10 titles like Mario Kart and Pokemon. On simpler games, agents get close to the human world record. However, on more complicated games like Pokemon Blue, the best models Kimi-K3 and Opus 5 are ~4x off the world record. Benchmark, paper, and demo below. @PatronusAI