Paweł Huryn(@PawelHuryn):Turns out DeepSeek harness improves the score of DeepSeek V4.1 Flash by 2-3 points on average. It also costs 80% more due to subagents and more output tokens. New results: DeepSeek V4.1 Flash 1017 + Claude Code harness: - max, mean of 7: 39.1 - high, mean of 5: 36.4 - low, mean of 5: 34.4 DeepSeek V4.1 Flash 1017 + DeepSeek harness: - max, mean of 7: 42.7 - high, mean of 5: 39.2 - low, mean of 5: 34.8 For comparison: GPT-6 Astra, max: 45 Opus 5.5, max: 41.7 Muse Wonder Full 1.0, max: 42 It's a crazy model, and the only one catched cheating. All the results are available on the benchmark website.

2026.10.18 08:04

Turns out DeepSeek harness improves the score of DeepSeek V4.1 Flash by 2-3 points on average. It also costs 80% more due to subagents and more output tokens. New results: DeepSeek V4.1 Flash 1017 + Claude Code harness: - max, mean of 7: 39.1 - high, mean of 5: 36.4 - low, mean of 5: 34.4 DeepSeek V4.1 Flash 1017 + DeepSeek harness: - max, mean of 7: 42.7 - high, mean of 5: 39.2 - low, mean of 5: 34.8 For comparison: GPT-6 Astra, max: 45 Opus 5.5, max: 41.7 Muse Wonder Full 1.0, max: 42 It's a crazy model, and the only one catched cheating. All the results are available on the benchmark website.