vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
가입 March 2024
36 팔로잉 중 50.2K 팬
Shoutout to XGrammar team for the great collaboration on enabling reliable and performant tool calling for serving DeepSeek-v4.1 on vLLM!
🚀 More reliable agents with DeepSeek V4.1!
XGrammar brings strict tool calling to SGLang & vLLM through Structural Tags, enforcing tool argument schemas in DeepSeek’s native format.
See how the Structural Tag works 👇
Check out XGrammar 👇
더 보기
