註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
加入 March 2024
37 正在關注    54.1K 粉絲
One router, two minds 🧠⚡ Huge work from @XunzhuoLiu and the vLLM Semantic Router team on System Duo — fast decision models handle the blink-of-an-eye calls (triage, guard, verify, check), while the router sends deep reasoning to the right model only when it matters. All open source, all running on vLLM →
顯示更多
𝗠𝗮𝗸𝗲 𝘆𝗼𝘂𝗿 𝗮𝗴𝗲𝗻𝘁 𝘁𝗵𝗶𝗻𝗸 𝗳𝗮𝘀𝘁 𝗮𝗻𝗱 𝘀𝗹𝗼𝘄. System One decides in a blink. System Two reasons in depth. 𝗢𝗻𝗲 𝗿𝗼𝘂𝘁𝗲𝗿. 𝗧𝘄𝗼 𝗺𝗶𝗻𝗱𝘀. 𝗜𝗻𝘁𝗿𝗼𝗱𝘂𝗰𝗶𝗻𝗴 𝗦𝘆𝘀𝘁𝗲𝗺 𝗗𝘂𝗼 → #Jev# #SystemOne# #SystemTwo# #OpenSource#
顯示更多
0
10
78
11
轉發到社區