Andy Konwinski(@andykonwinski):this is cool. we said we're gonna start treating benchmarks like software, so instead of starting from scratch each new release, we'll bump versions frequently as we fix bugs, add new tasks, and retire saturated tasks. well here we go! tbench 4 is hot on the heels of tb3 and harbor hub has a bunch of cool tools that make it easy to understand what's new and why

Andy Konwinski

@andykonwinski

Co-founder of @laudeinstitute, @laudeventures, @databricks, @perplexity_ai

参加 July 2010

120 フォロー中 12.1K ファン

Andy Konwinski@andykonwinski

2026.08.29 03:49

this is cool. we said we're gonna start treating benchmarks like software, so instead of starting from scratch each new release, we'll bump versions frequently as we fix bugs, add new tasks, and retire saturated tasks. well here we go! tbench 4 is hot on the heels of tb3 and harbor hub has a bunch of cool tools that make it easy to understand what's new and why

Ryan Marten@ryan_marten

2026.08.29 02:17

We've pushed a version update to the Terminal-Bench dataset and leaderboard. Terminal-Bench 4.0 calibrates task resources (time, cpu, memory), implements task fixes, and removes saturated tasks.