vLLM(@vllm_project):Kimi K3 decode on NVIDIA GB300 NVL72: 842 tokens/s at batch 1 with DSpark 🚀 A unified CuTe DSL decode path fuses attention, MoE and communication, delivering 1.5–2× higher decode throughput in several tested configurations and up to 1.8× higher P90 interactivity on the @SemiAnalysis_ AgentX workload. The post covers the kernel work, the correctness and long-context evals behind it, and benchmark sweeps across batch sizes, context lengths and parallelism configurations. Full write-up: https://t.co/4JbC73yX76 Great work and collaboration with the @NVIDIA team!

vLLM

@vllm_project

A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!

가입 March 2024

36 팔로잉 중 52.8K 팬

vLLM@vllm_project

2026.10.18 01:55

Kimi K3 decode on NVIDIA GB300 NVL72: 842 tokens/s at batch 1 with DSpark 🚀 A unified CuTe DSL decode path fuses attention, MoE and communication, delivering 1.5–2× higher decode throughput in several tested configurations and up to 1.8× higher P90 interactivity on the @SemiAnalysis_ AgentX workload. The post covers the kernel work, the correctness and long-context evals behind it, and benchmark sweeps across batch sizes, context lengths and parallelism configurations. Full write-up: Great work and collaboration with the @NVIDIA team!