Same weights. $50 of harness search. 3x on Terminal-Bench. We optimized the OpenCode harness for GPT-OSS-20B (high reasoning) with a meta-harness: an agent that reads failures, patches the harness, and validates the improvement. The model never changed. On held-out Terminal-Bench 2.1, the optimized harness scored 14.8%, up from OpenCode's 4.8%. That is 3.08x, from $49.97 of API credits. Three harness fixes carried most of the gain: - i7, verify before stopping: the model often edited code and ended the session without testing it. The harness gives it another turn to run the code. - i14, continue announced actions: the model would say "now run X" and then stop. The harness keeps the session going so it runs X. - i23, repair malformed tool calls: one extra `]` made OpenCode reject otherwise valid commands. The harness repairs that clear-cut error. How the meta-harness ran: 1. Start: stock OpenCode v1.18.13 + GPT-OSS-20B scored 8.5% on the dev split. 2. Iterate: Claude Code (Opus 5 max) reads the failed dev trajectories and proposes one patch. We build it and run it once per dev task, paired against the current parent. 3. Validate: candidates that clear a pre-registered margin get a 5-trial validation. They become the new parent only if their validated score surpasses the current one. 4. Stop: the loop ended after 23 iterations and about 2.4k dev attempts, when the $50 search budget was spent. Takeaway: pre-training puts capability in the weights, and post-training makes that capability usable. Harness optimization continues the same work at inference time, turning the model's behavior into more completed tasks. Same weights, 3x the finished tasks.

Codex is overoptimised for large models: it ranks 2nd out of 10 for GLM 5.2 but drops to 9th place for Gemma-4! Almost all the effort in this field goes into tuning the weights. We wanted to know how much of the final number is decided by the harness wrapped around them instead, so we ran 10 coding agent harnesses against two models on SWE-bench Pro. A lot, it turns out. Swapping the harness moves pass@1 from 23% to 52% on GLM-5.2, and from 15% to 36% on Gemma 4 26B-A4B: a wider gap than most model releases buy you. And the ranking does not transfer. The rank correlation between the two models' harness leaderboards is -0.05, which is to say: none. Codex is not alone in this. Every harness shipped by a model vendor drops on the small model — Codex 2nd to 9th, Claude Code 3rd to 7th, Qwen Code 4th to 6th — while the model-agnostic ones climb: crush 7th to 1st, opencode 8th to 2nd, pi 9th to 4th. The clearest case is crush, 7th on GLM-5.2 and 1st on Gemma 4. Run the identical scaffold on both and the small model wins by 4 points at a twelfth of the price, $0.30 per task against $3.61. Gemma 4's best harness beats GLM-5.2's four worst. A 26B model in the right scaffold is not far off a 744B model in the wrong one. Cost per solved task: $0.84 for Gemma 4 + crush at 36%. The cheapest GLM-5.2 setup that scores as well is openclaw at 38%, for $7.05. Output tokens per task span 16k to 621k across harnesses: a 39x spread in what you pay for, buying a 2x spread in what you get. 97% of input tokens are re-sent conversation prefix, so prompt caching is super important. The setup: every harness run against both models on the same 250 SWE-bench Pro tasks, one rollout each, priced at list API rates for the tokens each one actually spent. In the plot, a dark ring means the pairing sits on the Pareto frontier across both models and a faded dot means you could score higher for less elsewhere; 2 of the 10 harnesses (goose and hermes) are left out of the figure for legibility