If you retry a lot, Qwen3.8 27B can reach 92.04% on DeepSWE 1.1, ~18 pts above GPT-6 Astra’s reported ~74%.
I ran 20 configs across harnesses, thinking levels, and quantizations. Coverage varied a lot: two runs both scored 31.86% Reward, yet disagreed on 42/113 tasks. I saw this mainly for quantized models.
I pooled results per task: if any single candidate passed all new tests, the task passed. No patch mixing.
Best-of-k:
k=1 → 81.43% F2P / 37.21% reward (that's the average single-run performance)
k=10 → 98.31% / 86.25%
k=20 → 99.18% / 92.04%
This is oracle-selected, cross-config coverage, not single-run accuracy.
It shows a 27B model can produce passing solutions for most tasks. The harder problem is reliability + selecting the right candidate.
With strong independent acceptance + regression tests, retries to improve the result are less unrealistic: generate → verify → retry → stop on pass.
Cost is also important: imagine if one Qwen attempt costs 1/20 of a frontier attempt, 20 retries ≈ 1 frontier attempt in generation cost. But that’s only a hypothesis until actual per-attempt cost, verification, infra, and latency are measured.