oMLX 0.7.1.dev1 is out!
This dev release adds decision models (Clef, OpenJev), batched prefill for concurrent requests, a refreshed web UI, and faster Qwen3.8, Qwen3.6, GLM-5.3-Flash and DeepSeek V4.1.
Speedups in this release
- Qwen3.6-35B-A3B decode on M5 Max: 156 -> 205 tok/s (+31%).
- GLM-5.3-Flash decode on M3 Ultra: 31.3 -> 40.6 tok/s (+30%). The fused decode kernels now run on M3 and M4, not only M5.
- Qwen3.8-Flash-Next with expert offload, 32K prefill on M3 Ultra: 396 -> 636 tok/s (+61%).
- Qwen3.8-Flash-Next, 8 prompts arriving 0.4 s apart on M5 Max: time to first token 7.41 -> 5.75 s (-22%) with batched prefill.
What's new
- Decision models. Clef, Clef-Flash and OpenJev answer typed questions with a probability for each option through the new /v1/systemone endpoint.
- Batched prefill. Prompts that prefill at the same time now share one forward pass, so MoE models read their expert weights once.
- A refreshed web UI with a log table, a settings rail and a customizable dashboard.
- Native video input for Qwen3.5, 3.6 and 3.8.
- More accurate oQe quantization, with 11-15% lower KL to the original model.
- Fewer memory guard rejections in long sessions.
Most of the speedups came from contributors again. Thanks to everyone who sent PRs and tested the dev builds!