Register and share your invite link to earn from video plays and referrals.

Jun Kim
@jundotkim
Open-Source ML Engineer at ๐Ÿค— Hugging Face | Creator of oMLX | I build the tools I wish existed for my Mac, then open-source them. junkim.dot@gmail.com
Joined February 2026
198 Following    4.7K Followers
Thanks for the huge effort on this! Nice to see oMLX oQ quants holding up well on agentic tasks! ๐Ÿš€
Qwen3.8-Flash-Next tool-calling showdown: 9 setups, 5 inference engines, 92 agentic scenarios. It took a lot of time, enjoy it and share your feedback or requests for more tests ๐Ÿ”ฅ ๐Ÿ† Best score: oMLX oQ4e-MTP, 92/100 (@jundotkim) โšก Fastest + most deployable: mlx-serve mixed 4/8-bit, 1.5s/turn (@ddalcu) โšก๏ธ Best on DGX Spark Qwen3.8-Flash-Next-NVFP4-DGX-Spark (@ToNYD2WiLD) Score ยท median turn 92 oMLX oQ4e-MTP ยท 2.8s 90 vLLM NVFP4 ยท 2.7s 90 DwarfStar GGUF Q4 ยท 3.2s 90 vLLM NVFP4 2ร—Spark ยท 4.4s 89 mlx-serve 4/8-bit ยท 1.5s 89 TensorFold 4bit-MTP Spark ยท 3.3s 87 TensorFold oQ8-MTP ยท 3.2s 86 TensorFold 4bit-MTP ยท 1.9s 85 DwarfStar GGUF Q2 ยท 2.8s Takeaways: - Top 6 within 3 pts: speed decides - oMLX: best score and fewest safety fails (2) - DwarfStar Q2 squeezes into 64GB and still scores 85 ๐Ÿ”ฅ - 8-bit โ‰  better: oQ8 scored 87, safety-capped to โ˜…โ˜…โ˜… ๐Ÿ‘€ - 7 of 9 setups leaked another tenant's data (TC-92) Method: tool-eval-bench v2.7.1, 92 scenarios, thinking on, temperature 0, one run per setup, so treat 1โ€“2 point gaps as noise. by @SeraAndroid ๐Ÿ™ Note: I will retry with suggested Qwen settings for temperature & C. because maybe some engines are not considering the temp 0. Hardware used: - M3 Ultra 512GB - Single DGX Spark - Dual DGX Sparks Thanks @kernelpool @antirez @ToNYD2WiLD @MiaAI_lab @ashxhart @ddalcu @jundotkim
Show more