註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | 🔔 Follow for AI & Vibe Coding Tips 👇
加入 July 2023
551 正在關注    12.3K 粉絲
🍎 SGLang's main inference stack now runs on Apple Silicon. Previously, running SGLang on a Mac meant using its separate MLX backend. With v0.5.22, Apple Silicon now gets a standard PyTorch/MPS backend too. That means Macs can now run through SGLang's main pipeline ⚙️ scheduler 🧠 model runner 💾 memory-pool system 🌐 serving stack while MLX remains available as another backend. And we already have a first benchmark. On an M4 Pro with 48GB unified memory, Qwen3-0.6B in an offline batch-4 test reached: ⚡ 65.9 tps median decode 🚀 2,531 tps prefill On an M4 Pro with 48GB unified memory, SGLang reports Qwen3-0.6B at: ⚡ 65.9 tps median decode 🚀 2,531 tps prefill The Mac support also understands Apple's unified-memory limits properly. SGLang now plans memory around the amount Metal will actually allow. Link in ALT.
顯示更多