vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
加入 March 2024
36 正在关注 50.4K 粉丝
⚡ New RTX local-agent optimizations include a vLLM speedup on Blackwell.
@NVIDIARTXSpark reports:
🛠️ 1.2x vLLM performance on RTX PRO 6000 Blackwell
⚡ Up to 1.4x on a two-system DGX Spark cluster
Great to see the local serving path getting faster across RTX and DGX Spark.
显示更多
