It's only been a day since 0.7.1, but oMLX 0.7.2 is out with changes worth shipping now.
This release brings faster DeepSeek V4.1 on every oQ build, Lightning MTP in distributed inference, and Qwen3.8-Flash-Next GGUFs on Splash 1.4.0.
Faster DeepSeek V4.1 on oQ2e to oQ4e
- oQ3e with Lightning MTP + CED: decode 49.4 -> 68.2 tok/s (+38%), prefill 700 -> 949 tok/s (+36%). Every quant from oQ2e to oQ4e gets faster, with similar prefill gains.
- Prefill holds around 900 tok/s all the way to a 128K prompt.
- Measured on M3 Ultra 512GB. I'll run more tests on M5 Ultra 256GB.
Lightning MTP in Distributed Inference
- DeepSeek V4.1 and GLM-5.3 run Lightning MTP across Macs with both PP and TP.
- With TP on two M3 Ultras, both models decode more than 50% faster, even over a TCP ring. I'll share exact numbers after more tests over RDMA.
- A pipeline stall is fixed, so DeepSeek V4.1 across two Macs went from about 1 tok/s to about 28 tok/s.
More Models on Splash
- Qwen3.8-Flash-Next GGUFs run on Splash 1.4.0, and oMLX offers the matching DFlash2 draft.
- Split GGUFs download and show up as one model, and mlx-community OptiQ builds now start.
Other Improvements
- Better prompt cache reuse in Claude Code sessions, on both the built-in engines and Splash.
- Reasoning models answer Claude Code's auto-mode classifier before it times out.
- DFlash works with ParoQuant Qwen models.
- SpecPrefill no longer disturbs other requests decoding next to it, and Gemma 3 gets the right positions in sparse prefill.
- Keyboard access to the web UI menus and better contrast in the dark theme.
Thanks to everyone who sent PRs and tested 0.7.1 so quickly!