oMLX 0.7.3 is out! Ask AI, the question bar for your local models in any app, now talks back. Press Option-Shift-Space, speak, and your model answers out loud with captions. It can also search the web, read links and look at your screen. Ask AI voice and tools - Voice runs on LFM3-Omni, LFM3-Audio and Qwen3-Omni. With LFM3-Omni 2B on an M3 Ultra, the first audio comes about 100 ms after you stop talking. - Cut in while it's speaking, or share your screen in the middle of a conversation. - Web search, link reading and screen capture work in both text and voice. - Prompt placeholders like {date}, {app} and {clipboard}. Faster DeepSeek V4.1 - INT8 Activation Prefill on M5 Ultra: 64K prefill 2,882 -> 3,507 tok/s (+22%), 195K 2,476 -> 3,200 tok/s (+29%). - 12 concurrent requests decode 56.3 -> 125.7 tok/s (+123%). - Engram prefetch adds another +15% to prefill at 128K. Many requests at once, much less memory - 8 Lightning MTP streams at 81K on Qwen3.8-Flash-Next: peak memory 248 -> 161 GiB (-35%), decode 180.7 -> 241.1 tok/s (+33%). - Batched decode on oQ models: 16 requests 146.9 -> 221.9 tok/s (+51%) on M3 Ultra, 8 requests +74% on M2 Max. - Hybrid Qwen models reuse a shared system prompt across conversations, TTFT 0.89 -> 0.25 s. Also in 0.7.3 - Update downloaded models by fetching only the files that changed. - Better oQ quants, KL drops 9-49% for under 5% more size. - LFM3 text, audio and d1 models, and Unlimited-OCR-2. - Temporary chats in the web UI. Thanks to everyone who sent PRs and tested the rc!

oMLX 0.7.1 is out! This is the official 0.7.1 release. It adds GGUF models through Splash, much faster DeepSeek V4.1, 4-bit / 8-bit KV cache formats, and distributed inference for GLM-5.3 and DeepSeek V4.1. What's new - GGUF models through Splash. The downloader filters repos by MLX or GGUF and lets you pick the files, and Qwen3.8-27B and Qwen3.6-35B-A3B GGUFs run on Splash. - Affine4 and Affine8 KV cache. Full-attention KV is 74% smaller with Affine4 at 32K context. - A fix for GPU hangs when several models run at once. - Distributed inference for more models, GLM-5.3 and DeepSeek V4.1 now run across Macs with both pipeline (PP) and tensor (TP) parallelism. - Claude Desktop support. One toggle maps claude-opus, claude-sonnet and claude-haiku to your local models. - Realtime mic transcription with every STT model. Speedups in this release - DeepSeek V4.1 Flash decode on M3 Ultra: 20.9 -> 34.3 tok/s (+64%) for one request, 19.4 -> 91.8 tok/s (+373%) with 8 concurrent requests. - Qwen3.8-Flash-Next with Lightning MTP on M5 Max: decode holds 87-98 tok/s instead of swinging between 64 and 93. - Qwen3.8-Flash-Next with expert offload on M1 Max: +5-8% decode. Coming from 0.7.0? You also get Muse Spark Open, the Recommended models tab, DFlash in the batched engines, decision models, batched prefill and the improved oQe quantization. Thanks to everyone who sent PRs and tested the dev and rc builds!

oMLX 0.7.1.dev2 is out! oMLX can now serve Qwen3.8 and Qwen3.6 through Splash as an external engine. Splash runs the model on its own Metal engine with DFlash2, and the model still shows up in oMLX's model list, API, dashboard, cache stats and benchmarks like any other. This release also helps first-time users find a model that fits their Mac without guesswork, reworks model settings and chat in the web UI and the macOS app, and speeds up GLM-5.3 and DFlash. Splash as an external engine - Turn it on per model in Model Settings > Advanced. oMLX finds the Splash launcher and hands it the model and DFlash2 draft from your model directories, so nothing extra is downloaded. - Memory and context window are fitted to your oMLX memory limit, and an MLX model can run alongside it. - Vision checkpoints take images too. Speedups in this release - GLM-5.3-Flash decode on M5 Ultra: 85.5 -> 95.3 tok/s (+11%) at 64K context. - GLM-5.3-Flash, 4 streams on a shared 32K prompt on M5 Ultra: 81 -> 91 tok/s (+12%). - DFlash with 4 concurrent requests on M3 Ultra: +82% to +157% (MiMo V2.6 Flash, Qwen3-4B, Muse Glimmer 30B). What's new - Find the right model for your Mac. New users no longer have to guess which model to start with. The new Recommended tab shows only oQ models that will actually load on your Mac, with Lightning MTP on by default. - Reworked model settings. The web UI groups them into sections with a side menu, and the macOS app splits them into pages that save as you edit. - A new chat composer. Pick the model, profile and reasoning level right where you type, with a reasoning slider that follows the levels each model accepts. - DFlash now runs inside the batched engines, so it gets continuous batching and the prefix cache. DSpark drafters are accepted 61-85% of the time instead of 16-21%. - Requests survive memory pressure. They are paused and resumed instead of aborted, and cancelled prompts stay cached. - /v1/decisions for any chat model, plus the Decision 3.0 models with image and video input. - YaRN for Qwen3.5-family models past the native 262K context. - An "Allow inference without API key" toggle for LAN setups, and a Mistral Vibe launcher. - oQ for DeepSeek V4.1 at oQ2 to oQ2.7 and for EmbeddingGemma 2, plus MiniCPM-V 4.7 support. - A log table in the macOS app, copy buttons for model names, and Downloaded badges in the model browser. Most of the speedups came from contributors again. Thanks to everyone who sent PRs and tested the dev builds!

oMLX 0.7.1.dev1 is out! This dev release adds decision models (Clef, OpenJev), batched prefill for concurrent requests, a refreshed web UI, and faster Qwen3.8, Qwen3.6, GLM-5.3-Flash and DeepSeek V4.1. Speedups in this release - Qwen3.6-35B-A3B decode on M5 Max: 156 -> 205 tok/s (+31%). - GLM-5.3-Flash decode on M3 Ultra: 31.3 -> 40.6 tok/s (+30%). The fused decode kernels now run on M3 and M4, not only M5. - Qwen3.8-Flash-Next with expert offload, 32K prefill on M3 Ultra: 396 -> 636 tok/s (+61%). - Qwen3.8-Flash-Next, 8 prompts arriving 0.4 s apart on M5 Max: time to first token 7.41 -> 5.75 s (-22%) with batched prefill. What's new - Decision models. Clef, Clef-Flash and OpenJev answer typed questions with a probability for each option through the new /v1/systemone endpoint. - Batched prefill. Prompts that prefill at the same time now share one forward pass, so MoE models read their expert weights once. - A refreshed web UI with a log table, a settings rail and a customizable dashboard. - Native video input for Qwen3.5, 3.6 and 3.8. - More accurate oQe quantization, with 11-15% lower KL to the original model. - Fewer memory guard rejections in long sessions. Most of the speedups came from contributors again. Thanks to everyone who sent PRs and tested the dev builds!

oMLX 0.7.0rc1 is out! This release brings faster Qwen prefill & generation, MiMo V2.6, Ternary Bonsai 2, and partial block caching. DFlash now handles concurrent requests together, and Lightning MTP gets faster batch decoding! Performance on M5 Max, 128 GB (Prefill, oQ4e quant) - Qwen3.8-Flash-Next: 1,522 -> 2,007 tok/s (+32%) at 16K context. (Decode, batch=4, oQ4e quant) - Qwen3.8-27B with DFlash2: 56.9 -> 131.5 tok/s (+131%) - Qwen3.8-27B with Lightning MTP: 88.9 -> 136.9 tok/s (+54%) Full benchmark details are in the release notes. New models and features - MCDMA RDMA support for Mac + CUDA deployments, contributed by @ashxhart. - Partial block caching. No more reprocessing thousands of tokens just because they didn't fill a complete cache block. In one test, next-turn prefill dropped from 1,174 tokens to 37. - Ternary Bonsai 2 text and vision support. - MiMo V2.6 image, video, and audio understanding, plus Lightning MTP and DFlash for compatible checkpoints. - Broader MoE expert offload, including Lightning MTP alongside expert offload for DeepSeek V4.1 and GLM-5.3-Flash. This RC also includes the improvements from the dev releases, including Cluster v2, one-click model settings from community benchmarks, and a customizable dashboard. The GDN prefill kernels are adapted from @ddalcu's excellent mlx-serve! After a short round of testing, I'll publish the stable release and keep moving forward!

oMLX 0.7.0.dev4 adds DeepSeek V4.1 CED prefill with up to 79% faster prompt processing on M3 Ultra (opt-in), multi-request Lightning MTP, and plenty of quality-of-life improvements including one-click model settings. You can now apply model settings with one click using over 450,000 community benchmarks on Pick a top result for your Mac chip and model to use the same settings. Using a customized model? Copy a one-line recipe from a model with the same architecture and paste it into oMLX. There's also a customizable dashboard, global settings reset, and many bug fixes. See the release notes for the full details. Thank you for contributing code and sharing your benchmarks. Your benchmarks now help other users find and apply settings for their models with one click! * This release upgrades mlx-lm and mlx-vlm, with extensive internal changes. If something that worked before breaks, please open a GitHub issue with logs, and roll back to dev2 for now. ** I test as much as I can before each release, but I can't cover every Mac configuration and model. Issue reports on dev builds are a huge help to me.

oMLX community benchmark is back up, home to the largest collection of Apple Silicon MLX benchmark results! I got way more submissions than I expected, and my old plan couldn't keep up. So I improved the search logic and upgraded the plan. Depending on how traffic goes I may need to tune it more, but for now I'm just praying it holds 🙏 I'm also thinking about ways oMLX and the community benchmark can work better together. Stay tuned!

oMLX hits 47 tokens per second on a base M2 MacBook Pro by offloading context to the SSD. We explore how native MLX features achieve 3x faster generation than LM Studio in our latest test.

OML LB got me like: One wrong spam and poof - deranked to rat tourist status . Pro move: Thoughtful engages + original shitposts on BTC hustles. Gated Rune via RadFi, 1B supply - join before the streets overflow. @__O_M_L__ #OML#