Qwen3.8-Flash-Next tool-calling showdown: 9 setups, 5 inference engines, 92 agentic scenarios.
It took a lot of time, enjoy it and share your feedback or requests for more tests 🔥
🏆 Best score: oMLX oQ4e-MTP, 92/100 (
@jundotkim)
⚡ Fastest + most deployable: mlx-serve mixed 4/8-bit, 1.5s/turn (
@ddalcu)
⚡️ Best on DGX Spark Qwen3.8-Flash-Next-NVFP4-DGX-Spark (
@ToNYD2WiLD)
Score · median turn
92 oMLX oQ4e-MTP · 2.8s
90 vLLM NVFP4 · 2.7s
90 DwarfStar GGUF Q4 · 3.2s
90 vLLM NVFP4 2×Spark · 4.4s
89 mlx-serve 4/8-bit · 1.5s
89 TensorFold 4bit-MTP Spark · 3.3s
87 TensorFold oQ8-MTP · 3.2s
86 TensorFold 4bit-MTP · 1.9s
85 DwarfStar GGUF Q2 · 2.8s
Takeaways:
- Top 6 within 3 pts: speed decides
- oMLX: best score and fewest safety fails (2)
- DwarfStar Q2 squeezes into 64GB and still scores 85 🔥
- 8-bit ≠ better: oQ8 scored 87, safety-capped to ★★★ 👀
- 7 of 9 setups leaked another tenant's data (TC-92)
Method: tool-eval-bench v2.7.1, 92 scenarios, thinking on, temperature 0, one run per setup, so treat 1–2 point gaps as noise.
by
@SeraAndroid 🙏
Note: I will retry with suggested Qwen settings for temperature & C. because maybe some engines are not considering the temp 0.
Hardware used:
- M3 Ultra 512GB
- Single DGX Spark
- Dual DGX Sparks
Thanks
@kernelpool @antirez @ToNYD2WiLD @MiaAI_lab @ashxhart @ddalcu @jundotkim