Register and share your invite link to earn from video plays and referrals.

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | 🔔 Follow for AI & Vibe Coding Tips 👇
Joined July 2023
551 Following    12.3K Followers
🍎 SGLang's main inference stack now runs on Apple Silicon. Previously, running SGLang on a Mac meant using its separate MLX backend. With v0.5.22, Apple Silicon now gets a standard PyTorch/MPS backend too. That means Macs can now run through SGLang's main pipeline âš™ī¸ scheduler 🧠 model runner 💾 memory-pool system 🌐 serving stack while MLX remains available as another backend. And we already have a first benchmark. On an M4 Pro with 48GB unified memory, Qwen3-0.6B in an offline batch-4 test reached: ⚡ 65.9 tps median decode 🚀 2,531 tps prefill On an M4 Pro with 48GB unified memory, SGLang reports Qwen3-0.6B at: ⚡ 65.9 tps median decode 🚀 2,531 tps prefill The Mac support also understands Apple's unified-memory limits properly. SGLang now plans memory around the amount Metal will actually allow. Link in ALT.
Show more