Novita AI
@novita_labs
AI & Agent Cloud for Developers
โกJoin our community:
Joined September 2023
415 Following 5.7K Followers
Great work by the vLLM team! Happy to have supported these optimization efforts with compute. ๐
Kimi K3 serving in vLLM now delivers 2.2โ2.8x throughput on our B300 benchmark vs v0.27.1.
We break down the work across scheduling, KDA state handling, and MoE kernels, with benchmarks and commands to reproduce the results.
Thanks to the vLLM community for pushing Kimi K3 performance forward!
Read the deep dive:
Show more
