🎉 Great work from the PAN 2 team on an open-weight world model built for real-world simulation, grounded in time, physics and space to predict how scenes evolve as actions unfold. vLLM-Omni brings Day-0 support for PAN 2 text-to-video and image-to-video generation, making it available through an open inference stack.

vLLM × PyTorch: with the LibTorch Stable ABI, new PyTorch releases no longer force a vLLM rebuild 📷 Users upgrade sooner, distributors rebuild less, devs test torch changes without recompiling vLLM. Read the migration story:

Love seeing what the community builds on vLLM-Omni 🤩 OpenDirector: a local short-film workspace — generate, edit, export — powered by vLLM-Omni with a wide range of models. Congrats @hsliuunc! 🎬 🎥 OpenDirector: ⚡ vLLM-Omni:

One router, two minds 🧠⚡ Huge work from @XunzhuoLiu and the vLLM Semantic Router team on System Duo — fast decision models handle the blink-of-an-eye calls (triage, guard, verify, check), while the router sends deep reasoning to the right model only when it matters. All open source, all running on vLLM →

Kimi K3.1 Pro on TPUs, with day-0 support ⚡ Run it on TPU v7x with vllm-torchtpu. Great to see the continued collaboration between the @Google and @inferact teams! 🤝 Read more 👇

Thanks to @SemiAnalysis_ for highlighting the vLLM ecosystem’s work on TPU, in particular the Kimi K3 megakernel and vLLM TorchTPU powering AgentX results across GLM-5.2, Qwen3.5 397B, and Kimi K3. Shoutout to @inferact and @googlecloud for pushing vLLM performance on TPU. The @SemiAnalysis_ write-up below is a great read 👇

We’re excited to feature Kinference on the vLLM blog. @BuoyantIO built a new open source control plane for running vLLM on Kubernetes. Try it with the getting started guide and send the team your feedback!

🎉 Day-0 support for @Baidu_Inc’s ERNIE 5.5 Flash Base with vLLM! 280B total parameters, 14B active per token. Pre-trained on 33.5T tokens, with 128K training context and up to 1M tokens for fine-tuning and inference. Open Base weights for domain adaptation, instruction tuning and RL research. Code highlights: 82.3% HumanEval+ and 40.1% LiveCodeBench pass@1 (both 1-shot), versus 69.5% and 31.9% for DeepSeek-V4-Flash-Base under the same evaluation setup. Efficiency is built into the architecture: 37 compressed-attention layers + 6 MLA-DSA layers, Multi-Router Latent MoE, and ECHO — reusing the final layer as a speculative draft. Serve FP8 or BF16 with Baidu’s vLLM-ERNIE plugin on vLLM 0.32.0. ECHO runs through vLLM’s MTP path; recipes include expert parallelism, KV offload and prefill/decode disaggregation, with runs verified on @NVIDIA GB200 and H100. Weights are on @huggingface and @ModelScope2022. Recipe: FP8: BF16:

Hola Madrid! 🇪🇸 vLLM is coming to town — join us Nov 12 with @RedHat_AI and @NVIDIAAI for an evening of open-source inference: vLLM, vLLM-omni, LLM Compressor, and llm-d on Kubernetes. Register:

vLLM v0.33.0 is out! 🚀 927 commits · 389 contributors · 122 first-timers 🤖 New models: Reflection Beam-1.0, Step-5 Preview, Nemotron Parse 3.5, MiniCPM-OCR 1.0 🔥 PyTorch 2.14 — CUDA 13.0-only wheels, libtorch ABI-stable 🎲 Spec decoding: XPress refiner, FLy lossy verify (1.04–1.31x), DSpark 1373→3390 tok/s on GB300 ⚡ NVIDIA decode: 1.5–2x latency decode on GB300, FA4 sparse MLA, 39–48% lower TTFT 🚀 New AWS EFA image + shorter vllm/vllm name 🐧 ROCm 10.0 default, MonoKernel 2.32x P50 Full notes ➡️

Singapore, let's talk inference. Join the vLLM meetup with engineers from @inferact, @NVIDIA, @alibaba_cloud and the vLLM-Omni community: high-throughput prefill, RL, Blackwell, Vera Rubin, agent infrastructure and interactive world models. Nov 13 · 1–5:30 PM SGT · SQ Collective Free, with registration approval. Bring your serving questions. Meet the people building the stack.

Great work from the vLLM Semantic Router, KR Labs, and @AMD teams on Vela 3.0! Routing decisions and span extraction in one call, with six open models from 0.3B to 27B. More flexibility in what the router can detect and how small it can run.

🎉 Spark-X2.6-1.7B from @SparkLLM is out — a compact on-device agent model with 1M native context and controllable thinking (none/low/high), nearly matching its 4B sibling on SWE-Verified and τ²-bench. Day-0 vLLM support via trust-remote-code, incl. vLLM-Ascend for NPU deployment.

Great work from @AMD and the vLLM Semantic Router team on d3-ultra! Open multimodal decision-making with per-answer probabilities, visual grounding, and video tracking—trained and served on AMD Instinct GPUs. Apache-2.0 weights on @huggingface.

Beyond matching DeepInfer's performance, we also put vLLM to the test on @SemiAnalysis_ AgentX with real-world agentic workloads. At DeepSeek's off-peak pricing, vLLM delivers $38.96/GPU-hour revenue, with 954K tokens/s/GPU total throughput (cached+input+output) at 196 tokens/s P90 per user. This is the best DeepSeek-V4.1-Flash result on AgentX ever, powered by recent advancement in vLLM! Check out our blog for the full optimization story and how to reproduce the results 👇 Enjoy the read.

Scaled out with P/D disaggregation and Wide-EP on a GB300 NVL72 rack, vLLM serves V4.1-Flash on SemiAnalysis AgentX at 954K tokens/s per GPU with 196 tokens/s P90 per user, 7.9× our single-node result. Our engineering blog covers the work behind it:

vLLM matches DeepInfer, DeepSeek's own engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration across batch sizes 8 to 256, and does it at DEP32 against DeepInfer’s EP64DP64. DeepInfer was originally built on vLLM. By integrating DeepSeek's open-source kernels and extensively optimizing prefill and decode, vLLM now matches DeepInfer's performance curves.

Great to see MiniMax M3.1 Flash running in production at scale, served by @inferact with guaranteed SLAs — and the engineering improvements flowing back upstream to vLLM. This is what the open-source inference ecosystem is all about. Welcome @MiniMax_AI! 📷

🚀 Day-0 vLLM support for Sage-A1.5 (35B MoE, 3B active) from Shanghai AI Lab — agentic intelligence you can serve locally today: vllm serve InternScience/Sage-A1.5 Congrats @sageai_official ! 🔥