๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | ๐Ÿ”” Follow for AI & Vibe Coding Tips ๐Ÿ‘‡
๊ฐ€์ž… July 2023
552 ํŒ”๋กœ์ž‰ ์ค‘    12.5K ํŒฌ
A new Mac Inference engine brings a 48GB M4 Pro MacBook running Qwen3.6-35B to 154 tps. The same benchmark with llama.cpp + MTP, 69 tps. Someone built another Local AI inference engine for Mac. It's called Kapitsa, and instead of trying to support every model, it specializes using๐ŸŽ Apple Silicon for either ๐Ÿง  Qwen ๐ŸŸข NVIDIA Nemotron On a 2024 M4 Pro MacBook Pro with 48GB RAM, running Qwen3.6-35B-A3B at 4-bit: โšก Kapitsa + DFlash2 โ†’ 154 tps โšก oMLX + DFlash2 โ†’ 127 tps โšก Ollama + MTP โ†’ 89 tps โšก LM Studio โ†’ 86 tps โšก MTPLX โ†’ 80 tps ๐ŸŒ llama.cpp + MTP โ†’ 69 tps That's about 2.2ร— llama.cpp's decode speed in their test. The strategy being, instead of building one generic runtime, Kapitsa hand-optimizes Metal kernels + execution paths around specific architectures, then stacks tweaks like ๐Ÿš€ DFlash2 speculative decoding ๐Ÿง  model-aware caching โš™๏ธ optimized MoE execution ๐Ÿ”€ concurrent request batching And on an M5 Max 128GB, they report ๐Ÿš€ Qwen3.6-35B โ†’ 281 tps ๐Ÿ‘๏ธ Nemotron 3.5 Super-VL 120B โ†’ 85 tps โš ๏ธ These are still developer benchmarks, not independent results. Links in ALT.
๋” ๋ณด๊ธฐ