vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
参加 March 2024
37 フォロー中 54.1K ファン
Vela 2.0 brings span-level decisions to routing: safety checks, domain classification, PII spans and unsupported claims in one call.
Built by vLLM Semantic Router and KR Labs. Four sizes, 0.3B to 9B. Apache-2.0.
もっと見る
