Register and share your invite link to earn from video plays and referrals.

vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
Joined March 2024
37 Following    54.1K Followers
vLLM v0.33.0 is out! ๐Ÿš€ 927 commits ยท 389 contributors ยท 122 first-timers ๐Ÿค– New models: Reflection Beam-1.0, Step-5 Preview, Nemotron Parse 3.5, MiniCPM-OCR 1.0 ๐Ÿ”ฅ PyTorch 2.14 โ€” CUDA 13.0-only wheels, libtorch ABI-stable ๐ŸŽฒ Spec decoding: XPress refiner, FLy lossy verify (1.04โ€“1.31x), DSpark 1373โ†’3390 tok/s on GB300 โšก NVIDIA decode: 1.5โ€“2x latency decode on GB300, FA4 sparse MLA, 39โ€“48% lower TTFT ๐Ÿš€ New AWS EFA image + shorter vllm/vllm name ๐Ÿง ROCm 10.0 default, MonoKernel 2.32x P50 Full notes โžก๏ธ
Show more