vLLM(@vllm_project):vLLM hit new peak bs=1 decode on Kimi-K3: 464 tok/s 🚀 Under a low-entropy reasoning workload, Kimi-K3 + DSpark on vLLM reaches 464 tok/s on batch size 1 with 4×4 GB300. This benchmark is fully reproducible with public image: vllm/vllm-openai:kimi-k3 and @inferact's DSpark draft model linked in the thread. 1/3

vLLM

@vllm_project

A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!

加入 March 2024

36 正在關注 46.7K 粉絲

vLLM@vllm_project

2026.07.29 00:49

vLLM hit new peak bs=1 decode on Kimi-K3: 464 tok/s 🚀 Under a low-entropy reasoning workload, Kimi-K3 + DSpark on vLLM reaches 464 tok/s on batch size 1 with 4×4 GB300. This benchmark is fully reproducible with public image: vllm/vllm-openai:kimi-k3 and @inferact's DSpark draft model linked in the thread. 1/3