Register and share your invite link to earn from video plays and referrals.

Jun Kim
@jundotkim
Open-Source ML Engineer at 🤗 Hugging Face | Creator of oMLX | I build the tools I wish existed for my Mac, then open-source them. junkim.dot@gmail.com
Joined February 2026
198 Following    4.7K Followers
oMLX 0.7.1 is out! This is the official 0.7.1 release. It adds GGUF models through Splash, much faster DeepSeek V4.1, 4-bit / 8-bit KV cache formats, and distributed inference for GLM-5.3 and DeepSeek V4.1. What's new - GGUF models through Splash. The downloader filters repos by MLX or GGUF and lets you pick the files, and Qwen3.8-27B and Qwen3.6-35B-A3B GGUFs run on Splash. - Affine4 and Affine8 KV cache. Full-attention KV is 74% smaller with Affine4 at 32K context. - A fix for GPU hangs when several models run at once. - Distributed inference for more models, GLM-5.3 and DeepSeek V4.1 now run across Macs with both pipeline (PP) and tensor (TP) parallelism. - Claude Desktop support. One toggle maps claude-opus, claude-sonnet and claude-haiku to your local models. - Realtime mic transcription with every STT model. Speedups in this release - DeepSeek V4.1 Flash decode on M3 Ultra: 20.9 -> 34.3 tok/s (+64%) for one request, 19.4 -> 91.8 tok/s (+373%) with 8 concurrent requests. - Qwen3.8-Flash-Next with Lightning MTP on M5 Max: decode holds 87-98 tok/s instead of swinging between 64 and 93. - Qwen3.8-Flash-Next with expert offload on M1 Max: +5-8% decode. Coming from 0.7.0? You also get Muse Spark Open, the Recommended models tab, DFlash in the batched engines, decision models, batched prefill and the improved oQe quantization. Thanks to everyone who sent PRs and tested the dev and rc builds!
Show more