Qwen3.8-Flash-Next tool-calling showdown: 9 setups, 5 inference engines, 92 agentic scenarios.
It took a lot of time, enjoy it and share your feedback or requests for more tests ๐ฅ
๐ Best score: oMLX oQ4e-MTP, 92/100 (
@jundotkim)
โก Fastest + most deployable: mlx-serve mixed 4/8-bit, 1.5s/turn (
@ddalcu)
โก๏ธ Best on DGX Spark Qwen3.8-Flash-Next-NVFP4-DGX-Spark (
@ToNYD2WiLD)
Score ยท median turn
92 oMLX oQ4e-MTP ยท 2.8s
90 vLLM NVFP4 ยท 2.7s
90 DwarfStar GGUF Q4 ยท 3.2s
90 vLLM NVFP4 2รSpark ยท 4.4s
89 mlx-serve 4/8-bit ยท 1.5s
89 TensorFold 4bit-MTP Spark ยท 3.3s
87 TensorFold oQ8-MTP ยท 3.2s
86 TensorFold 4bit-MTP ยท 1.9s
85 DwarfStar GGUF Q2 ยท 2.8s
Takeaways:
- Top 6 within 3 pts: speed decides
- oMLX: best score and fewest safety fails (2)
- DwarfStar Q2 squeezes into 64GB and still scores 85 ๐ฅ
- 8-bit โ better: oQ8 scored 87, safety-capped to โ
โ
โ
๐
- 7 of 9 setups leaked another tenant's data (TC-92)
Method: tool-eval-bench v2.7.1, 92 scenarios, thinking on, temperature 0, one run per setup, so treat 1โ2 point gaps as noise.
by
@SeraAndroid ๐
Note: I will retry with suggested Qwen settings for temperature & C. because maybe some engines are not considering the temp 0.
Hardware used:
- M3 Ultra 512GB
- Single DGX Spark
- Dual DGX Sparks
Thanks
@kernelpool @antirez @ToNYD2WiLD @MiaAI_lab @ashxhart @ddalcu @jundotkim