vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
๊ฐ์
March 2024
36 ํ๋ก์ ์ค 50.4K ํฌ
โก New RTX local-agent optimizations include a vLLM speedup on Blackwell.
@NVIDIARTXSpark reports:
๐ ๏ธ 1.2x vLLM performance on RTX PRO 6000 Blackwell
โก Up to 1.4x on a two-system DGX Spark cluster
Great to see the local serving path getting faster across RTX and DGX Spark.
๋ ๋ณด๊ธฐ
