vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
参加 March 2024
37 フォロー中 54.1K ファン
vLLM × PyTorch: with the LibTorch Stable ABI, new PyTorch releases no longer force a vLLM rebuild 📷 Users upgrade sooner, distributors rebuild less, devs test torch changes without recompiling vLLM. Read the migration story:
もっと見る
