@googlecloud and Inferact are announcing today a partnership to make TPU a first-class citizen in @vllm_project. This partnership puts both teams on one engineering roadmap to bring TPU to the broader open model ecosystem, optimizing vLLM as the agentic production serving engine for TPU: • Production serving features and optimized kernels • A native PyTorch path via TorchTPU • Moving towards day-0 support for frontier model releases We're also launching a community program: shared TPU capacity for open-source contributors, plus dedicated review and design help from the core vLLM maintainers at Inferact. Everything this collaboration produces is open source. Read the full announcement:

Sparse MLA only attends to the top-K tokens, so the rest of the KV need not live on the GPU. Hybrid HiSparse in vLLM builds on that, and a request keeps decoding after its KV stops fitting in HBM. It keeps KV on the GPU while there is room. Under pressure a request releases its coldest pages to host memory, keeps a small hot buffer of what the indexer asks for, and keeps decoding instead of being preempted. 📊 Demonstrated on GLM 5.3, one 8× H200 node, full 1M context. Same host memory, configured concurrency 32: KV offloading kept 5-6 requests running. Hybrid HiSparse kept 19-25. 🔹 Hot pages are ordinary KV blocks from the same pool (Hybrid Memory Allocator) 🔹 One fused kernel resolves resident, hot and missing rows, CUDA-graph capturable 🔹 Prefix caching, OffloadingConnector, P/D imports and MTP keep working Built by @RedHat_AI and @PrimeIntellect with the vLLM community. Planned for v0.30; pinned commit, flags and calculator are in the post👇 🔗

🫡 vLLM is your engine of choice for agentic workload. The pareto curve is easily understood; but very hard to optimize against. The @inferact team did amazing work here, please check it out (and join us!)

New blog is out: vLLM x AgentX: Optimizing for Real-World Agentic Serving. Agent traffic stresses every layer of the serving stack at once. This post walks the full-stack work for optimizing vLLM on Agentic workloads, including the architecture, framework, and runtime optimizations, measured on AgentX, @SemiAnalysis_'s public agentic benchmark. 🧵1/6

I couldn't find a public performant MXFP8 GEMM on MI355X, so I just asked @KimiDevs K3 to write one, and it beats torch._scaled_mm by >2x times 🤯 This is an adaptation of the official Gluon MXFP4 example. FlyDSL adaptation is WIP. @AnushElangovan you guys should try K3 for kernel engineering if you haven't!

Congratulations to @HUMAIN and @MiniMax_AI on HUMAIN-M3, a frontier Arabic model now available on HUMAIN Node 🎉. @vllm_project is running the inference under the hood, and we're looking forward to more Arabic use cases with this state-of-the-art stack.

🎬Video generation faster than playback! 🚀MiniMax H3 on vLLM-Omni + FastVideo's FastH3: a complete 10.1s MP4 - video AND synchronized audio - rendered in 8.7s!⚡️ Thanks to @MiniMax_AI for the great Minimax H3 release, the FastVideo team @haoailab for open-sourcing FastH3 and helping on the serving integration, and @NVIDIAAI for the continued sponsorship and joint optimization efforts!

Personally I believe open research that benefits the broader OSS community deserves encouragement, not dismissal. Academia → production is one of the hardest jumps in MLSys and the FastVideo team did it with limited compute. Efforts like this deserve support, not trash talk.

(1/6) Open Weight @MiniMax_AI FastH3 v1: Generate 15s 768p video in 13s 🚀 - FastVideo collab w/ @nuvalab + FastGen - Up to 14x speedup on @NVIDIAAI Blackwell GPU - Fully open so community can run and improve the acceleration recipe. The era of open weight video models just started, that calls for post training. With great community effort such as Minimax H3 Max based on @MiniMax_AI , we have more to share about how post training can improve speed, quality and most importantly, how it’s done, with open weights and recipes.

Qwen3.8-Flash-Next from @Alibaba_Qwen has day-0 support in vLLM, verified on NVIDIA and AMD GPUs. 🎉 Ultra-sparse multimodal MoE: 125B params, 6B active, 262K native, 1M via YaRN. On top of those sits a separate 51B N-gram table you can offload. Most of it will look familiar. The Gated DeltaNet layers reuse the KV path vLLM has had since Qwen3-Next: only a quarter of the layers hold a growing KV cache. Keep the 51B table in host RAM instead of HBM with VLLM_PLE_CPU_OFFLOAD=1. Qwen Sparse Attention is where the new engine work went. For now the model runs from vllm/vllm-openai:qwen38-flash-next. Thanks to @Alibaba_Qwen for the weights, and for opening them this early! 🙌 🔗

🫡 AgentX 1.0 from @SemiAnalysis_ has been months in the making, and the @inferact team did amazing work pushing @vllm_project to become the leading inference system for agents: KV offload, PD disaggregation, and production-grade reliability — all without compromising accuracy!