Register and share your invite link to earn from video plays and referrals.

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | ๐Ÿ”” Follow for AI & Vibe Coding Tips ๐Ÿ‘‡
Joined July 2023
552 Following    12.5K Followers
A new Mac Inference engine brings a 48GB M4 Pro MacBook running Qwen3.6-35B to 154 tps. The same benchmark with llama.cpp + MTP, 69 tps. Someone built another Local AI inference engine for Mac. It's called Kapitsa, and instead of trying to support every model, it specializes using๐ŸŽ Apple Silicon for either ๐Ÿง  Qwen ๐ŸŸข NVIDIA Nemotron On a 2024 M4 Pro MacBook Pro with 48GB RAM, running Qwen3.6-35B-A3B at 4-bit: โšก Kapitsa + DFlash2 โ†’ 154 tps โšก oMLX + DFlash2 โ†’ 127 tps โšก Ollama + MTP โ†’ 89 tps โšก LM Studio โ†’ 86 tps โšก MTPLX โ†’ 80 tps ๐ŸŒ llama.cpp + MTP โ†’ 69 tps That's about 2.2ร— llama.cpp's decode speed in their test. The strategy being, instead of building one generic runtime, Kapitsa hand-optimizes Metal kernels + execution paths around specific architectures, then stacks tweaks like ๐Ÿš€ DFlash2 speculative decoding ๐Ÿง  model-aware caching โš™๏ธ optimized MoE execution ๐Ÿ”€ concurrent request batching And on an M5 Max 128GB, they report ๐Ÿš€ Qwen3.6-35B โ†’ 281 tps ๐Ÿ‘๏ธ Nemotron 3.5 Super-VL 120B โ†’ 85 tps โš ๏ธ These are still developer benchmarks, not independent results. Links in ALT.
Show more