oMLX 0.7.0rc1 is out! This release brings faster Qwen prefill & generation, MiMo V2.6, Ternary Bonsai 2, and partial block caching. DFlash now handles concurrent requests together, and Lightning MTP gets faster batch decoding! Performance on M5 Max, 128 GB (Prefill, oQ4e quant) - Qwen3.8-Flash-Next: 1,522 -> 2,007 tok/s (+32%) at 16K context. (Decode, batch=4, oQ4e quant) - Qwen3.8-27B with DFlash2: 56.9 -> 131.5 tok/s (+131%) - Qwen3.8-27B with Lightning MTP: 88.9 -> 136.9 tok/s (+54%) Full benchmark details are in the release notes. New models and features - MCDMA RDMA support for Mac + CUDA deployments, contributed by @ashxhart. - Partial block caching. No more reprocessing thousands of tokens just because they didn't fill a complete cache block. In one test, next-turn prefill dropped from 1,174 tokens to 37. - Ternary Bonsai 2 text and vision support. - MiMo V2.6 image, video, and audio understanding, plus Lightning MTP and DFlash for compatible checkpoints. - Broader MoE expert offload, including Lightning MTP alongside expert offload for DeepSeek V4.1 and GLM-5.3-Flash. This RC also includes the improvements from the dev releases, including Cluster v2, one-click model settings from community benchmarks, and a customizable dashboard. The GDN prefill kernels are adapted from @ddalcu's excellent mlx-serve! After a short round of testing, I'll publish the stable release and keep moving forward!

My GitHub account has been restored and the oMLX repo is accessible again. GitHub said it was flagged by their automated abuse detection and cleared after manual review. everything should be back to normal. Thanks everyone for the kind messages and patience today!

I'm happy to announce that I've joined Hugging Face. What started as a personal project back in February is now something I get to work on full time. Local AI has grown explosively this year. I've said this since the early oMLX releases: I want my friend who bought a MacBook yesterday to be able to run AI on it today. I believe MLX has that potential. Apple Silicon is the easiest entry point for a regular person to get started with AI, with no complicated hardware to assemble. And the open source community, including Hugging Face, has been growing that potential. That support has already made a huge difference. Hugging Face is the best place for me to support oMLX and the MLX community with everything I have. For anyone getting into local AI, the first step usually starts at Hugging Face. Mine did too. I'm proud that I get to work at that entry point, where new ideas can be tried out. oMLX stays exactly where it is, under the same Apache 2.0 license in the same repository, and I'll keep leading the project, same as before. What changes is that I can now spend far more time on it, move faster, and build something sustainable for the long term together with all the contributors who have put so much into it. I also want to do more for MLX as a whole. oMLX is built on top of transformers, mlx-lm, mlx-vlm and the rest of the ecosystem, and I'm grateful to the people behind them. Rather than keeping everything inside oMLX, over time I want to push work upstream where it makes sense. And where the community needs something that doesn't exist yet, oMLX is a good place to try it first. Thank you to the 264 contributors who have built oMLX with me, and to everyone who filed issues with detailed logs and reproductions so we could fix things. oMLX would not be what it is without you. oMLX continues in the same place, in the same way. Just much faster. HF's announcement:

oMLX 0.7.0.dev4 adds DeepSeek V4.1 CED prefill with up to 79% faster prompt processing on M3 Ultra (opt-in), multi-request Lightning MTP, and plenty of quality-of-life improvements including one-click model settings. You can now apply model settings with one click using over 450,000 community benchmarks on Pick a top result for your Mac chip and model to use the same settings. Using a customized model? Copy a one-line recipe from a model with the same architecture and paste it into oMLX. There's also a customizable dashboard, global settings reset, and many bug fixes. See the release notes for the full details. Thank you for contributing code and sharing your benchmarks. Your benchmarks now help other users find and apply settings for their models with one click! * This release upgrades mlx-lm and mlx-vlm, with extensive internal changes. If something that worked before breaks, please open a GitHub issue with logs, and roll back to dev2 for now. ** I test as much as I can before each release, but I can't cover every Mac configuration and model. Issue reports on dev builds are a huge help to me.

The latest oMLX release by @jundotkim is terrific. Finally can use a local model (qwen3.8-flash-next-q4) running on my M3 Ultra Mac Studio, from my iPhone (with Open Minis) to do "real" assistant work. Running at a respectable ~60tps. No cloud costs. Love it.

oMLX community benchmark is back up, home to the largest collection of Apple Silicon MLX benchmark results! I got way more submissions than I expected, and my old plan couldn't keep up. So I improved the search logic and upgraded the plan. Depending on how traffic goes I may need to tune it more, but for now I'm just praying it holds 🙏 I'm also thinking about ways oMLX and the community benchmark can work better together. Stay tuned!

oMLX 0.5.2 is out. (Sorry for the long silence!) oMLX is the most convenient way to run MLX models on your Mac, and the fastest way to run them, with custom Metal kernels for GLM, MiniMax, DeepSeek V4, and Qwen. New in 0.5.2: - Live activity in the macOS menu bar: opt-in readouts for live, session, and all-time throughput (PP and TG), plus CPU/GPU/MEM usage bars, each with a popover and rolling graphs - A reorganized Models menu: load models directly from the menu bar, sectioned into Loaded, Favorites, and Library with load state and sizes - Bonsai 1-bit / 2-bit decode kernels for extreme low-bit models - Faster Hugging Face downloads, more TTS output formats, and chat history import/export And if you are coming from 0.4.x, here is what the 0.5 line added: - oMLX custom kernels: prefill improved by up to +45% for DeepSeek-V4-Flash, +33% for Qwen3.6-27B, +99% for GLM-5.2, and +94% for MiniMax M3 on my M3 Ultra - Lightning MTP: native speculative decoding that accelerates TG on top of the custom-kernel PP path. Qwen3.6-35B-A3B went from 89.6 to 140.4 tok/s and Qwen3.6-27B from 35.0 to 55.1 tok/s - oQe imatrix enhanced quantization: The highest-accuracy quantization on MLX, driven by an activation-importance calibration pass - Plus a lot of stability and memory hardening across 0.5.1 and 0.5.2 for long-running servers Huge thanks to everyone who contributed, tested, reported bugs, and shared feedback. I hope this makes running local models on your Mac even nicer!