Agent performance isn't just about the model — the harness design matters just as much.
TL;DR
JIT-Agent dynamically synthesizes, repairs, and evolves agent harnesses (scaffolds) based on task characteristics at runtime. It achieves average gains of +7.7pt on GLM-5.2 and +8.8pt on DeepSeek-V4-Flash, reaching top-1 performance on 8 of 9 benchmarks — surpassing GPT-5.6 and all tested frontier models.
Title: Scaling Harness Intelligence via Just-in-Time Harness Evolution
URL:
Key Points
🧩 Harnesses formalized as machine-learnable artifacts
The four-module protocol h = (M, P, A, F) — Memory, Planning, Action, Capability Orchestration — constrains the generation space while remaining expressive enough to represent all 13 harnesses in HarnessFactory.
🎓 Three-stage training: imitation → repair → evolution
Stage I learns from teacher-generated harnesses; Stage II trains recovery from execution failures (max 2 iterations); Stage III's Evo-GDPO evolves harnesses that advance the Pareto frontier on performance, latency, and cost simultaneously.
📊 Higher accuracy AND lower cost at the same time
On xBench-DeepSearch: score 78→82 (+4pt), tokens 527K→212K (▲60%), cost $0.075→$0.039 (▲48%). Average 36% token reduction versus best fixed harness across all 9 benchmarks.
⚡ Transfers across model families without retraining
JIT-generated harnesses outperform ReAct on DeepSeek V4 (+10.2pt avg), Mimo V2.5 (+8.6pt), and Qwen 3.6 (+4.0pt) — no need to retrain the harness generator for each backbone.
🔄 Online evolution continues improving at deployment
Streaming mode accumulates successful harnesses across task sequences, outperforming static generation on all three evaluated benchmarks.
"Harness intelligence" as a trainable scaling dimension orthogonal to model weights is the key conceptual contribution here.
#
AIAgents# #
LLMScaling#