๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
๊ฐ€์ž… March 2024
37 ํŒ”๋กœ์ž‰ ์ค‘    54.7K ํŒฌ
vLLM EPD, now upgraded ๐Ÿš€ DeepInfer demonstrated the value of EPD disaggregation. vLLMโ€™s latest EPD improvements deliver lower TTFT and higher throughput per GPU, counting all encoder GPUs, integrated with vLLM Router, SMG, and llm-d! Read more here!๐Ÿ‘‡
๋” ๋ณด๊ธฐ