vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
加入 March 2024
37 正在關注 54.1K 粉絲
We’re excited to feature Kinference on the vLLM blog. @BuoyantIO built a new open source control plane for running vLLM on Kubernetes.
Try it with the getting started guide and send the team your feedback!
顯示更多
Our friends at @vllm_project just published our post introducing Kinference, a new open source project. Kinference is an inference control plane for Kubernetes and vLLM. There’s a getting started guide at the end, so you can try it on your own cluster 👉
顯示更多
