The leading provider of enterprise open source solutions, using a community-powered approach to deliver Linux, hybrid cloud, edge, and Kubernetes technologies.
Scale distributed inference and control AI serving costs with confidence. llm-d 1.0 brings multi-tenant quality of service, zero-downtime rolling updates, and prefix-cache-aware routing to your production workloads. Explore the release: #PyTorchCon#