vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
Joined March 2024
37 Following 54.1K Followers
Vela 2.0 brings span-level decisions to routing: safety checks, domain classification, PII spans and unsupported claims in one call.
Built by vLLM Semantic Router and KR Labs. Four sizes, 0.3B to 9B. Apache-2.0.
Show more
