vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
Joined March 2024
36 Following 50.4K Followers
⚡ New RTX local-agent optimizations include a vLLM speedup on Blackwell.
@NVIDIARTXSpark reports:
🛠️ 1.2x vLLM performance on RTX PRO 6000 Blackwell
⚡ Up to 1.4x on a two-system DGX Spark cluster
Great to see the local serving path getting faster across RTX and DGX Spark.
Show more
