登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | 🔔 Follow for AI & Vibe Coding Tips 👇
参加 July 2023
551 フォロー中    12.3K ファン
🍎 SGLang's main inference stack now runs on Apple Silicon. Previously, running SGLang on a Mac meant using its separate MLX backend. With v0.5.22, Apple Silicon now gets a standard PyTorch/MPS backend too. That means Macs can now run through SGLang's main pipeline ⚙️ scheduler 🧠 model runner 💾 memory-pool system 🌐 serving stack while MLX remains available as another backend. And we already have a first benchmark. On an M4 Pro with 48GB unified memory, Qwen3-0.6B in an offline batch-4 test reached: ⚡ 65.9 tps median decode 🚀 2,531 tps prefill On an M4 Pro with 48GB unified memory, SGLang reports Qwen3-0.6B at: ⚡ 65.9 tps median decode 🚀 2,531 tps prefill The Mac support also understands Apple's unified-memory limits properly. SGLang now plans memory around the amount Metal will actually allow. Link in ALT.
もっと見る