vLLM(@vllm_project):The P/D prototype connected DEP4 prefill to DEP8 decode on 12× GB300 with FP8, NIXL and a 384 GiB cache store. It recorded 514,784 prefix-hit tokens (91.24%); two 1,000-request OCR runs scored 90.4% and 89.4%, with zero request errors. This was a pre-release, pre-rebase test. The P/D PR remains open; check the final build separately. 5/6

vLLM

@vllm_project

A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!

가입 March 2024

36 팔로잉 중 52.8K 팬

vLLM@vllm_project

2026.10.15 03:26

The P/D prototype connected DEP4 prefill to DEP8 decode on 12× GB300 with FP8, NIXL and a 384 GiB cache store. It recorded 514,784 prefix-hit tokens (91.24%); two 1,000-request OCR runs scored 90.4% and 89.4%, with zero request errors. This was a pre-release, pre-rebase test. The P/D PR remains open; check the final build separately. 5/6