vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
Joined March 2024
37 Following 54.1K Followers
We’re excited to feature Kinference on the vLLM blog. @BuoyantIO built a new open source control plane for running vLLM on Kubernetes.
Try it with the getting started guide and send the team your feedback!
Show more
Our friends at @vllm_project just published our post introducing Kinference, a new open source project. Kinference is an inference control plane for Kubernetes and vLLM. There’s a getting started guide at the end, so you can try it on your own cluster 👉
Show more
