vLLM(@vllm_project):Scaled out with P/D disaggregation and Wide-EP on a GB300 NVL72 rack, vLLM serves V4.1-Flash on SemiAnalysis AgentX at 954K tokens/s per GPU with 196 tokens/s P90 per user, 7.9× our single-node result. Our engineering blog covers the work behind it:

vLLM

@vllm_project

A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!

参加 March 2024

37 フォロー中 54.1K ファン

vLLM@vllm_project

2026.10.24 17:59

Scaled out with P/D disaggregation and Wide-EP on a GB300 NVL72 rack, vLLM serves V4.1-Flash on SemiAnalysis AgentX at 954K tokens/s per GPU with 196 tokens/s P90 per user, 7.9× our single-node result. Our engineering blog covers the work behind it: