vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
参加 March 2024
36 フォロー中 50.2K ファン
Shoutout to XGrammar team for the great collaboration on enabling reliable and performant tool calling for serving DeepSeek-v4.1 on vLLM!
🚀 More reliable agents with DeepSeek V4.1!
XGrammar brings strict tool calling to SGLang & vLLM through Structural Tags, enforcing tool argument schemas in DeepSeek’s native format.
See how the Structural Tag works 👇
Check out XGrammar 👇
もっと見る
