SGLang
@sgl_project
Run LLMs fast at any scale ๐
Join our community
For AI tech blogs & deep-dives ๐ @lmsysorg
44 Following 4.1K Followers
Google Cloud @googlecloud announced its Day 0 support for Kimi K3 with SGLang! They just published the full guide for serving the 2.8T-param MoE with SGLang on GKE, including DSPARK speculative decoding.
A model this size is an infrastructure problem before it's a model problem. A4 and A4X VMs gave SGLang the memory bandwidth and interconnect to keep K3 fast under real concurrency.
Three ways to deploy: Model Garden, AI Hypercomputer recipes, GKE.
Guide in the comments ๐
Show more
Congrats to @TencentHunyuan on the Hy3 release, Day-0 support in SGLang is ready.
Full launch command + details in cookbook:
