註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
加入 March 2024
37 正在關注    54.7K 粉絲
vLLM EPD, now upgraded 🚀 DeepInfer demonstrated the value of EPD disaggregation. vLLM’s latest EPD improvements deliver lower TTFT and higher throughput per GPU, counting all encoder GPUs, integrated with vLLM Router, SMG, and llm-d! Read more here!👇
顯示更多
0
21
124
22
轉發到社區