๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | ๐Ÿ”” Follow for AI & Vibe Coding Tips ๐Ÿ‘‡
๊ฐ€์ž… July 2023
551 ํŒ”๋กœ์ž‰ ์ค‘    12.3K ํŒฌ
๐ŸŽ SGLang's main inference stack now runs on Apple Silicon. Previously, running SGLang on a Mac meant using its separate MLX backend. With v0.5.22, Apple Silicon now gets a standard PyTorch/MPS backend too. That means Macs can now run through SGLang's main pipeline โš™๏ธ scheduler ๐Ÿง  model runner ๐Ÿ’พ memory-pool system ๐ŸŒ serving stack while MLX remains available as another backend. And we already have a first benchmark. On an M4 Pro with 48GB unified memory, Qwen3-0.6B in an offline batch-4 test reached: โšก 65.9 tps median decode ๐Ÿš€ 2,531 tps prefill On an M4 Pro with 48GB unified memory, SGLang reports Qwen3-0.6B at: โšก 65.9 tps median decode ๐Ÿš€ 2,531 tps prefill The Mac support also understands Apple's unified-memory limits properly. SGLang now plans memory around the amount Metal will actually allow. Link in ALT.
๋” ๋ณด๊ธฐ