TL;DR: Agents fail not from a lack of model smarts but from a weak execution harness. "Harness scaling" — hardening the runtime without touching weights — reaches 95.3% on Terminal-Bench 2.1 for ~$15.
Title: StateM: Reaching 95.3% Raw Accuracy, or a $15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling
URL:
Key points
🧠 Scale the harness, not the model — learn from failures with no retraining
📌 Durable states retain discoveries across episodes
🎯 Phase-local context surfaces only relevant info at decision points
✅ Checked transitions verify preconditions to stop cascading failures
📖 Recoverable runbooks turn failures into queryable knowledge
🕒 Versioned practices let agents update and roll back strategies
💰 GPT-5.6 hits 95.3% (445 trials, all 89 tasks), compressing a $574.68 run to ~$15
It turns every failure into an executable precondition, lifting reliability from the operational layer.
#
AIAgents# #
TerminalBench#