註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Zbigniew Majewski
@majewskizby
加入 May 2014
53 正在關注    508 粉絲
I increased test time 4 times (same 44 tasks, 4 agents in parallel, auto-graded): 🥇 DeepSeek-V4.1-Flash max + DeepSeek Harness: 41/44 in 87 min 🥈 DeepSeek-V4.1-Flash max + Terminus-2: 41/44 in 204 min (was 36) 🥉 DeepSeek-V4.1-Flash max + DeepSeek Harness + "work efficiently" prompt: 39/44 in 54 min What changed: • Max effort was mostly too slow, not too dumb. With more time it catches up. • Low effort gained nothing. Its misses are real errors, not timeouts. • The efficiency prompt increased speed but decreased quality: about 40% less time in both harnesses, but 2–3 fewer tasks solved. • The harness matters as much as the model: same DeepSeek, same 41/44, 204 min in Terminus-2 vs 87 min in DeepSeek Harness.
顯示更多
Time-limited agentic benchmark on DGX Spark (44 tasks, 4 agents in parallel, auto-graded): 🥇 DeepSeek-V4.1-Flash (low effort, 4 Sparks): 40/44 🥈 GLM-5.3 (low effort, 4 Sparks): 37/44 GLM-5.3-Flash (2 Sparks): 33/44 MiniMax-M3.1-Flash (4 Sparks): 26/44 (but best vision) Low effort edged out max for both DS and GLM. Agent tasks are time-bound, so thinking less finishes more - though max wasn't far behind. Caveat: these are practical, moderately hard tasks (TB 2.1 subset: 14 medium, 6 hard). On harder problems without a clock, higher effort may still win.
顯示更多