Register and share your invite link to earn from video plays and referrals.

Jun Kim
@jundotkim
Open-Source ML Engineer at 🤗 Hugging Face | Creator of oMLX | I build the tools I wish existed for my Mac, then open-source them. junkim.dot@gmail.com
Joined February 2026
198 Following    4.7K Followers
oMLX 0.7.4.dev1 is out! With the new Prefill Accelerator, connect an M5 Max to your big Mac and prefill up to 2.9x faster! Prefill Accelerator (64K prompt, M3 Ultra 512GB + M5 Max 128GB) - DeepSeek V4.1 Flash: 1,060 -> 3,123 tok/s (+195%) - Qwen3.8 Flash-Next: 1,309 -> 3,666 tok/s (+180%) - GLM-5.3 Flash: 540 -> 1,388 tok/s (+157%) - MiniMax M3.1: 297 -> 569 tok/s (+92%) - An M5 Max 64GB still adds 30-118%, and decode speed stays the same. Faster DeepSeek V4.1 with A8 and buffer pool improvements - Prefill on M5 Ultra is 43-48% faster, 32K goes 2,631 -> 3,779 tok/s (+44%). - oQ4e builds now run INT8 Activation Prefill on M5. Faster Qwen3.8 Flash-Next - FP16 builds get the fused kernels. Decode on M3 Ultra goes 38.9 -> 63.0 tok/s (+62%), and +23% on M2 Max. - Decode after long prompts is 3-5% faster on M5 Max. Better for agents - Parallel tool results from Claude Code reach the model in call order, so Qwen no longer pairs a result with the wrong call. - Agent sessions on a cluster reuse the prompt cache across tool turns, 743 -> 9,266 cached tokens by the fifth turn. - Switch oMLX models inside Codex and Copilot with /model. Also in 0.7.4.dev1 - Ming-3.0 TTS with a 2x faster decode loop. - Ask AI can use an oMLX server on another Mac. - Check for model updates in the macOS app. - German across the web dashboard and the macOS app. - Batched decode with 6 requests on 8-bit models is 16% faster. A big thank you to everyone who sent PRs, reported bugs and tested builds for this release, and a warm welcome to the five new contributors!
Show more