A new Mac Inference engine brings a 48GB M4 Pro MacBook running Qwen3.6-35B to 154 tps.
The same benchmark with llama.cpp + MTP, 69 tps.
Someone built another Local AI inference engine for Mac.
It's called Kapitsa, and instead of trying to support every model, it specializes using๐ Apple Silicon for either
๐ง Qwen
๐ข NVIDIA Nemotron
On a 2024 M4 Pro MacBook Pro with 48GB RAM, running Qwen3.6-35B-A3B at 4-bit:
โก Kapitsa + DFlash2 โ 154 tps
โก oMLX + DFlash2 โ 127 tps
โก Ollama + MTP โ 89 tps
โก LM Studio โ 86 tps
โก MTPLX โ 80 tps
๐ llama.cpp + MTP โ 69 tps
That's about 2.2ร llama.cpp's decode speed in their test.
The strategy being, instead of building one generic runtime, Kapitsa hand-optimizes Metal kernels + execution paths around specific architectures, then stacks tweaks like
๐ DFlash2 speculative decoding
๐ง model-aware caching
โ๏ธ optimized MoE execution
๐ concurrent request batching
And on an M5 Max 128GB, they report
๐ Qwen3.6-35B โ 281 tps
๐๏ธ Nemotron 3.5 Super-VL 120B โ 85 tps
โ ๏ธ These are still developer benchmarks, not independent results.
Links in ALT.