Guanlan Dai(@guanlan):We’re spending up to 100B tokens across 26,040 agent runs to answer one question: when you switch models, should you switch harnesses too? Our v1.0 FrontierHarness leaderboard ranked harnesses on a single model. V1.1 expands it to 21 harnesses × 5 models × 62 tasks × 4 attempts, 72x the size of v1.0. Results are coming in live. Link below🧵

Guanlan Dai

@guanlan

Building @Runta, the execution layer that controls what AI agents can actually do. Prev @Cloudflare and @Kong.

参加 January 2010

484 フォロー中 6.8K ファン

Guanlan Dai@guanlan

2026.11.10 16:31

We’re spending up to 100B tokens across 26,040 agent runs to answer one question: when you switch models, should you switch harnesses too? Our v1.0 FrontierHarness leaderboard ranked harnesses on a single model. V1.1 expands it to 21 harnesses × 5 models × 62 tasks × 4 attempts, 72x the size of v1.0. Results are coming in live. Link below🧵