🚀 vLLM-Omni v0.20.0 is out — aligned with upstream vLLM v0.20.0 (CUDA 13.0 · PyTorch 2.11 · Transformers 5.x). ⚡ Qwen3-Omni throughput +72% on H20, 32 conc (0.241 → 0.414 req/s) via talker / code2wav multi-replica scaling 🎙️ TTS faster & leaner: VoxCPM2 RTF 0.946 → 0.106 · Fish Speech Fast AR latency -53% · Qwen3-TTS / Voxtral-TTS Code2Wav saves ~3.2 GiB 🎨 Diffusion dynamic step-level batching: +7.8% throughput / -5.8% latency 🆕 New / improved: HunyuanImage-3.0, ERNIE T2I, AudioX, Wan2.2-S2V, LTX-2.3, FastGen Wan 2.1 📱 Wan2.2 on NPU production-ready: MindIE-SD, fused ops, VAE BF16, HSDP/USP — +50–60% perf 🧮 Quant expanded: Qwen Omni W4A16, OmniGen2 FP8, Z-Image FP8, HunyuanImage3 NPU, GLM-Image 🧩 Multi-backend updates across CUDA / ROCm / MUSA / NPU / XPU Check it out →

Connect with us at the vLLM Meetup Singapore! 🚀 Join engineers from NVIDIA, Inferact, Alibaba Cloud, and the vLLM-Omni community to explore the latest advances in AI serving, training, agent infrastructure, and real-time inference. Bring your toughest AI serving challenges and connect with the engineers building the stack. 📅 Nov. 13, 2026 | 1:00–5:30 PM SGT 📍 SQ Collective, Singapore 👉 Register free today:

🚀 Excited to share the vLLM-Omni technical report: a unified serving runtime for omni-modality generation. 📄 Paper: 🔗 Repo: Speech assistants, visual generation, world models, and robot loops have pushed serving past a single text decode loop. The execution patterns diverge with the output: multi-stage autoregressive pipelines, iterative diffusion, and sessions that carry state from step to step. LLM servers and diffusion stacks each go deep on only one of these, so deployments fall back to stitching disjoint runtimes together. vLLM-Omni is the shared control plane for that mix. An orchestrator advances each request across stages; specialized engines run the compute; a connector carries the payloads; and the same session path keeps duplex, world-model, and robot loops on one runtime. Really grateful to the vLLM and vLLM-Omni teams for the collaboration and support. Contributions are welcome. 🙏

🎉 Fish Audio Drama 3 is now supported in vLLM-Omni! Voice cloning, speaker tags and expressive delivery through /v1/audio/speech, with streaming 44.1 kHz audio. The model has 35B total parameters and 3B active per token. The merged optimization PR reports 128 concurrent streams on one H200: 255 ms median time to first audio and 222.2 seconds of audio generated per second. Both stages share the GPU; averaged over 5 runs of 1,088 Seed-TTS-Eval English requests per run, with pre-encoded voice references. Try it on current upstream: vllm serve fishaudio/drama-3 --omni --port 8091 Thanks to Fish Audio and the vLLM-Omni contributors! Recipe:

Singapore, let's talk inference. Join the vLLM meetup with engineers from @inferact, @NVIDIA, @alibaba_cloud and the vLLM-Omni community: high-throughput prefill, RL, Blackwell, Vera Rubin, agent infrastructure and interactive world models. Nov 13 · 1–5:30 PM SGT · SQ Collective Free, with registration approval. Bring your serving questions. Meet the people building the stack.

🎉 Day-0 support for FLUX 3 Image Klein is here in vLLM-Omni! We worked with Black Forest Labs and NVIDIA to bring the new image-model release to the vLLM-Omni serving stack. What the integration brings: 🎨 FLUX 3 Image Klein image generation ⚡ BF16, FP8 and NVFP4 checkpoint formats 🛠️ Custom kernels, with optimizations across the transformer and text encoder 📦 Hugging Face model-path handling for the text encoder and autoencoder Thanks to Black Forest Labs, NVIDIA and the vLLM-Omni community for the collaboration. Docker image: docker pull vllm/vllm-omni:flux-3-image-klein Explore vLLM-Omni: Deployment recipe:

Here is th eQwen-Image-2.1 official 8 steps turbo model. I know vllm-omni has been doing a lot of work to enable faster inference, but not sure about how it has been doing elsewhere. What's the fastest end to end solution now?

At PyTorch Conference North America, Ricardo Noriega de Soto, Tech Lead for the vLLM Omni team and Alexander Brooks, Principal Machine Learning Engineer @RedHat, will demonstrate how extending vLLM’s prefix caching mechanism to multistage pipelines boosts inference speeds while reducing GPU memory overhead. Join us in San Jose on October 20th to learn practical strategies for optimizing complex AI workloads: #PyTorchCon# @vllm_project

The MiniMax H3 ecosystem is growing faster than ever! ☺️ Check out Awesome MiniMax H3 Integrations - an index tracking everything built around H3, from 24GB VRAM local ComfyUI setups to enterprise-scale SGLang & vLLM-Omni serving deployments. 🔥 What’s inside: - Hardware & Precision Maps: Quantization guides across INT8, NVFP4, and GGUF down to 8GB VRAM limits - Speed & Serving: Acceleration LoRAs, Sol-Attn, block-caching, and multi-GPU production stacks - Developer Tooling: Native agent skills, timeline directors, multi-shot motion context nodes, and h3.c for Apple Silicon Huge thanks to the global open-source community, framework maintainers, and independent developers pushing the boundaries of what’s possible with MiniMax H3! 👇 Explore the repository & contribute:

MiniMax H3 (Hailuo 3) video on 2× DGX Spark 🎬 • 5 s 1344×768 clip with audio: 643 s (one Spark: 1399 s) • 10 s: 1690 s, 15 s: 3080 s (one Spark: out of memory) • 20 s: 6740 s, beyond the model's 15 s spec and still coherent to the last frame (clip below, one continuous generation, seed 1101, 20 steps) • Text-encoder TP2 + Ulysses SP2 over one ConnectX-7 cable (RoCE, 19.5 GB/s) • Online FP8 DiT, cuDNN attention, 24 fps, native audio • Frames leave the GPU as uint8, VAE tiles decoded on both GPUs: same bytes, 15 s fits with 25 GB to spare on each host • H3-Base open weights (768p), vLLM-Omni, no Ray and no changes to the vLLM-Omni tree vllm serve cannot span two hosts, so the recipe adds a cross-host executor: rank 1 runs on the second Spark and is driven over the link through vLLM's own message queues, behind the unchanged single-node API. Quality checks, memory traces and the open levers are in the repo. Full recipe, measurements and credits: