vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
加入 March 2024
36 正在關注 50.2K 粉絲
Shoutout to XGrammar team for the great collaboration on enabling reliable and performant tool calling for serving DeepSeek-v4.1 on vLLM!
🚀 More reliable agents with DeepSeek V4.1!
XGrammar brings strict tool calling to SGLang & vLLM through Structural Tags, enforcing tool argument schemas in DeepSeek’s native format.
See how the Structural Tag works 👇
Check out XGrammar 👇
顯示更多
