登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | 🔔 Follow for AI & Vibe Coding Tips 👇
参加 July 2023
552 フォロー中    12.5K ファン
A new Mac Inference engine brings a 48GB M4 Pro MacBook running Qwen3.6-35B to 154 tps. The same benchmark with llama.cpp + MTP, 69 tps. Someone built another Local AI inference engine for Mac. It's called Kapitsa, and instead of trying to support every model, it specializes using🍎 Apple Silicon for either 🧠 Qwen 🟢 NVIDIA Nemotron On a 2024 M4 Pro MacBook Pro with 48GB RAM, running Qwen3.6-35B-A3B at 4-bit: ⚡ Kapitsa + DFlash2 → 154 tps ⚡ oMLX + DFlash2 → 127 tps ⚡ Ollama + MTP → 89 tps ⚡ LM Studio → 86 tps ⚡ MTPLX → 80 tps 🐌 llama.cpp + MTP → 69 tps That's about 2.2× llama.cpp's decode speed in their test. The strategy being, instead of building one generic runtime, Kapitsa hand-optimizes Metal kernels + execution paths around specific architectures, then stacks tweaks like 🚀 DFlash2 speculative decoding 🧠 model-aware caching ⚙️ optimized MoE execution 🔀 concurrent request batching And on an M5 Max 128GB, they report 🚀 Qwen3.6-35B → 281 tps 👁️ Nemotron 3.5 Super-VL 120B → 85 tps ⚠️ These are still developer benchmarks, not independent results. Links in ALT.
もっと見る