Andy Konwinski(@andykonwinski):this is cool. we said we're gonna start treating benchmarks like software, so instead of starting from scratch each new release, we'll bump versions frequently as we fix bugs, add new tasks, and retire saturated tasks. well here we go! tbench 4 is hot on the heels of tb3 and harbor hub has a bunch of cool tools that make it easy to understand what's new and why

2026.08.29 03:49

this is cool. we said we're gonna start treating benchmarks like software, so instead of starting from scratch each new release, we'll bump versions frequently as we fix bugs, add new tasks, and retire saturated tasks. well here we go! tbench 4 is hot on the heels of tb3 and harbor hub has a bunch of cool tools that make it easy to understand what's new and why

Ryan Marten@ryan_marten

2026.08.29 02:17

We've pushed a version update to the Terminal-Bench dataset and leaderboard. Terminal-Bench 4.0 calibrates task resources (time, cpu, memory), implements task fixes, and removes saturated tasks.

轉發到社區