๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
๊ฐ€์ž… March 2024
36 ํŒ”๋กœ์ž‰ ์ค‘    45.4K ํŒฌ
With Kimi K3 Day-0 on vLLM: Open Frontier Intelligence for Everyone ๐Ÿš€ At 2.8 trillion parameters, Moonshot AI's Kimi K3 is one of the most powerful open-weight models ever released. Starting today, you can serve it on vLLM the moment the weights are public. What K3 brings: ๐Ÿง  2.8T-parameter Mixture-of-Experts (16 of 896 experts active per token) ๐Ÿ“š 1M-token context window ๐Ÿ› ๏ธ Native multimodal understanding, including vision โšก Kimi Delta Attention: a hybrid of linear and full attention that makes million-token context affordable Huge thank you to @Kimi_Moonshot AI for the model release and partnership, @inferact for leading the vLLM optimizations, and to our partners at @nvidia, @AMD, and the broader vLLM community. 1/6
๋” ๋ณด๊ธฐ