Novita AI
@novita_labs
AI & Agent Cloud for Developers
⚡Join our community:
加入 September 2023
415 正在關注 5.7K 粉絲
Great work by the vLLM team! Happy to have supported these optimization efforts with compute. 🚀
Kimi K3 serving in vLLM now delivers 2.2–2.8x throughput on our B300 benchmark vs v0.27.1.
We break down the work across scheduling, KDA state handling, and MoE kernels, with benchmarks and commands to reproduce the results.
Thanks to the vLLM community for pushing Kimi K3 performance forward!
Read the deep dive:
顯示更多
