登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

Jun Kim
@jundotkim
Open-Source ML Engineer at 🤗 Hugging Face | Creator of oMLX | I build the tools I wish existed for my Mac, then open-source them. junkim.dot@gmail.com
参加 February 2026
198 フォロー中    4.7K ファン
Thanks for the huge effort on this! Nice to see oMLX oQ quants holding up well on agentic tasks! 🚀
Qwen3.8-Flash-Next tool-calling showdown: 9 setups, 5 inference engines, 92 agentic scenarios. It took a lot of time, enjoy it and share your feedback or requests for more tests 🔥 🏆 Best score: oMLX oQ4e-MTP, 92/100 (@jundotkim) ⚡ Fastest + most deployable: mlx-serve mixed 4/8-bit, 1.5s/turn (@ddalcu) ⚡️ Best on DGX Spark Qwen3.8-Flash-Next-NVFP4-DGX-Spark (@ToNYD2WiLD) Score · median turn 92 oMLX oQ4e-MTP · 2.8s 90 vLLM NVFP4 · 2.7s 90 DwarfStar GGUF Q4 · 3.2s 90 vLLM NVFP4 2×Spark · 4.4s 89 mlx-serve 4/8-bit · 1.5s 89 TensorFold 4bit-MTP Spark · 3.3s 87 TensorFold oQ8-MTP · 3.2s 86 TensorFold 4bit-MTP · 1.9s 85 DwarfStar GGUF Q2 · 2.8s Takeaways: - Top 6 within 3 pts: speed decides - oMLX: best score and fewest safety fails (2) - DwarfStar Q2 squeezes into 64GB and still scores 85 🔥 - 8-bit ≠ better: oQ8 scored 87, safety-capped to ★★★ 👀 - 7 of 9 setups leaked another tenant's data (TC-92) Method: tool-eval-bench v2.7.1, 92 scenarios, thinking on, temperature 0, one run per setup, so treat 1–2 point gaps as noise. by @SeraAndroid 🙏 Note: I will retry with suggested Qwen settings for temperature & C. because maybe some engines are not considering the temp 0. Hardware used: - M3 Ultra 512GB - Single DGX Spark - Dual DGX Sparks Thanks @kernelpool @antirez @ToNYD2WiLD @MiaAI_lab @ashxhart @ddalcu @jundotkim
もっと見る