TL;DR: Agents fail not from a lack of model smarts but from a weak execution harness. "Harness scaling" โ hardening the runtime without touching weights โ reaches 95.3% on Terminal-Bench 2.1 for ~$15.
Title: StateM: Reaching 95.3% Raw Accuracy, or a $15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling
URL:
Key points
๐ง Scale the harness, not the model โ learn from failures with no retraining
๐ Durable states retain discoveries across episodes
๐ฏ Phase-local context surfaces only relevant info at decision points
โ
Checked transitions verify preconditions to stop cascading failures
๐ Recoverable runbooks turn failures into queryable knowledge
๐ Versioned practices let agents update and roll back strategies
๐ฐ GPT-5.6 hits 95.3% (445 trials, all 89 tasks), compressing a $574.68 run to ~$15
It turns every failure into an executable precondition, lifting reliability from the operational layer.
#
AIAgents# #
TerminalBench#