๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
๊ฐ€์ž… March 2024
37 ํŒ”๋กœ์ž‰ ์ค‘    54.1K ํŒฌ
One router, two minds ๐Ÿง โšก Huge work from @XunzhuoLiu and the vLLM Semantic Router team on System Duo โ€” fast decision models handle the blink-of-an-eye calls (triage, guard, verify, check), while the router sends deep reasoning to the right model only when it matters. All open source, all running on vLLM โ†’
๋” ๋ณด๊ธฐ
๐— ๐—ฎ๐—ธ๐—ฒ ๐˜†๐—ผ๐˜‚๐—ฟ ๐—ฎ๐—ด๐—ฒ๐—ป๐˜ ๐˜๐—ต๐—ถ๐—ป๐—ธ ๐—ณ๐—ฎ๐˜€๐˜ ๐—ฎ๐—ป๐—ฑ ๐˜€๐—น๐—ผ๐˜„. System One decides in a blink. System Two reasons in depth. ๐—ข๐—ป๐—ฒ ๐—ฟ๐—ผ๐˜‚๐˜๐—ฒ๐—ฟ. ๐—ง๐˜„๐—ผ ๐—บ๐—ถ๐—ป๐—ฑ๐˜€. ๐—œ๐—ป๐˜๐—ฟ๐—ผ๐—ฑ๐˜‚๐—ฐ๐—ถ๐—ป๐—ด ๐—ฆ๐˜†๐˜€๐˜๐—ฒ๐—บ ๐——๐˜‚๐—ผ โ†’ #Jev# #SystemOne# #SystemTwo# #OpenSource#
๋” ๋ณด๊ธฐ