LLM Compressor v0.14.0 is out, and GPTQ just got its biggest speedup since launch. A new Triton kernel makes quantization ~15x faster end to end. Batching layers that share a shape pushes that to ~30x on some MoE workloads. Even the old eager path is 1.5-2x faster. Also new: expanded MSE/iMatrix observers that beat GPTQ for NVFP4 on internal benchmarks, REAP pruning with distributed DDP, and support for GLM 5.3 and Qwen3.8. Full release notes:

Training and rollout logprobs matched bit for bit on ROCm. The @RLKernel team integrated RL-Align/RL-Kernel with vllm-project/vime. A 200-step Qwen3-8B GRPO run on 8× @AMD MI300X recorded zero logprob mismatches between Megatron training and vLLM rollout. The strict path aligns reduction order, intermediate precision, rounding points, and math primitives across both sides. Deep dive:

Open-source AI is moving fast. There’s a lot to unpack as workloads become increasingly agentic. Great to have @KaichaoYou share his perspective at #ApsaraConference2026#. Thanks @alibaba_cloud for having us. Excited to keep pushing open-source inference forward together. 🚀

The @inferact team open sourced a TPU megakernel for Kimi K3 achieving 709 tokens/s, against 450 tokens/s on GB200. All 92 of K3's MoE layers run in a single Pallas kernel, with weight prefetching that reaches across layer boundaries so transfers for one layer overlap with computation in the previous one. Shoutout to the team! Writeup and repo link in the thread.

DiffusionGemma-Jev now runs on vLLM 🚀 Ask yes/no, multiple-choice, or scored questions and get confidence with every answer. vLLM seeds a canvas with the response template, leaves only the answer slots noisy, then reads a probability distribution from every slot in a single denoising step. Huge thanks to @mmastrac for driving this upstream! 🙏

762 commits. 315 contributors. 104 first-timers. vLLM v0.30.0 is live. 🎉 Highlights: 🤖 Hybrid-attention hot paths: Kimi K3 streamlines KDA, AttnRes, and MLA; DeepSeek-V4.1-Flash adds MXFP8 KV and async Engram; Qwen3.8-Flash-Next fuses QSA/PLE and cuts sparse-GQA overhead 🗄️ HiSparse adds a host tier beneath sparse-MLA decode; under GPU pressure, only top-k misses return to a per-request hot buffer 🛠️ Model Runner V2 brings EAGLE3-style drafts to pipeline parallelism and extends adaptive verification to every draft-model speculator through online acceptance estimation (#50514#, #52228#) 🖋️ Dual-key Gumbel-max watermark generation and detection, with per-request opt-out and speculative-decoding support 🆕 New models include GLM-5.3-Flash, K2-Horizon, Cohere Compass, and Bailing V3 VL ⚡ Fast Start keeps post-quantized, TP-sharded weights in a per-GPU daemon; restarts map them over CUDA IPC with --load-format ipc_cache Thread 👇

Excited to see @peano_ai run full-parameter RL for @XiaomiMiMo 310B MiMo-V2.6 on TPUs, across 1,000+ TPUs. 🚀🚀 vLLM drives the rollouts, bitwise-matched with the trainer in validation. All 310B params move across the ICI fabric in under 2s. 🔗

🎉 Congrats to @XiaomiMiMo on MiMo-V2.6: day-0 support in vLLM for both sizes. Two MoEs with text, image, video and audio in one checkpoint: Pro is 1.02T total with 42B active, Flash is 309B with 15B active. 1M context, native FP8 weights. Built-in DFlash spec decoding drafts 7 tokens a step. Same architecture class as V2.5, and the serving pieces carry over: hybrid sliding-window plus global attention, the mimo reasoning and tool-call parsers, DFlash spec decoding. Thanks to the @XiaomiMiMo team for opening the weights and the RL stack! 🙌 🔗

vLLM integrated PyNvVideoCodec to offload video decoding from CPU to GPU's NVDEC. Result: 2x+ throughput at 8×H100, CPU bottleneck gone! This is huge for video captioning at scale (AV training, metadata), and ships with CUDA vLLM releases. 🔗

Qwen3.8-2.4T on vLLM: a Pareto frontier spanning 5K total tokens/s/GPU at high throughput and 180 output tokens/s/user at low latency, across tuned PD configurations on @nvidia GB300 NVL72. Workload: 8K input / 1K output. Drawing on lessons from trial and error, we walk through the tuning decisions step by step: budget KV cache, benchmark prefill and decode separately, then choose topologies and MTP settings for each serving target. Deployment configs are included so you can reproduce the results. Great work from the @NVIDIAAI contributors and the vLLM community! Explore the frontier:

Qwen-Image-2.1 from @Alibaba_Qwen has day-0 support in vLLM-Omni. 🎉 Generate, edit, and create transparent images with one model: a 7.1B DiT paired with Qwen3-VL-8B. Setup and supported combinations 👇

Thank you to everyone who showed up, spoke, and stayed throughout the vLLM Conference. Lines out the door for talks say a lot about this community. Thank you to @inferact for hosting! Session recordings are in the thread.

Kimi K3 serving in vLLM now delivers 2.2–2.8x throughput on our B300 benchmark vs v0.27.1. We break down the work across scheduling, KDA state handling, and MoE kernels, with benchmarks and commands to reproduce the results. Thanks to the vLLM community for pushing Kimi K3 performance forward! Read the deep dive:

Everything you need to start self-hosting an open LLM. Run it on your own hardware. No API keys. No per-token bill. Nothing leaves your machine. The full path with @vllm_project: batch inference in Python, an OpenAI-compatible API server in one command, and quantized models that cut an 8B from ~16GB of weights to a quarter of that while keeping 98-100% accuracy. Walkthrough by @cedricclyburn.

🚀 vLLM's Humming backend can run Chord, @novita_labs' open-source W4A16 MoE kernels for Kimi K2.x. Kernel gains reach 1.33x on H200 TP8 vs tuned public Humming and 2.15x on B300 EP8 decode vs its untuned default. The indexed path works on compatible vLLM revisions; grouped integration is WIP. Great to see the kernels and benchmarks open-sourced! Details:

What does it take to serve a model that talks back, or one that generates video with audio? Most text models advance one token at a time. These don't, and they need different scheduling to match. New recap on how vLLM-Omni serves them:

@googlecloud and Inferact are announcing today a partnership to make TPU a first-class citizen in @vllm_project. This partnership puts both teams on one engineering roadmap to bring TPU to the broader open model ecosystem, optimizing vLLM as the agentic production serving engine for TPU: • Production serving features and optimized kernels • A native PyTorch path via TorchTPU • Moving towards day-0 support for frontier model releases We're also launching a community program: shared TPU capacity for open-source contributors, plus dedicated review and design help from the core vLLM maintainers at Inferact. Everything this collaboration produces is open source. Read the full announcement:

🚀 Day-0 support for @intern_lm Intern-S2-397B is now available in vLLM! Built for long-horizon scientific research, Intern-S2 brings strong multimodal, reasoning, coding, and scientific agent capabilities. Run it with vLLM today 👇 🔗