We’re releasing speculative decoding in our inference engine Uzu, starting with Qwen3.6-27B.
On Apple M5 Max with 128 GB of unified memory, our Mirai-M stack reaches 105 output tokens/sec entirely on-device - 2.9× faster than the fastest MLX speculative-decoding implementation we benchmarked.
The result comes from full-stack co-design: DFlash + our Weaver model, tree-based speculative decoding, Mirai quantization, our verification algorithm, and Metal kernels optimized for Apple silicon.
Mirai builds the full stack for frontier on-device intelligence. 105 t/s is the evidence, full-stack integration is the moat. But tokens/sec are not actually the metric we ultimately care about.
More on that soon.
Full benchmarks + methodology: