So Gartner's Lovelock will take a big swipe at data centers, beware, after that could be smooth sailing. But wait until hoe vociferous he is before you buy your next quarter
717 commits. 307 contributors. 96 first-timers. vLLM v0.31.0 is live. 🎉
Highlights:
🤖 DeepSeek-V4.1-Flash: FlashMLA mega attention with NVFP4 KV caching, DeepGEMM sparse MQA, Mega-Gate, and shared Engram host tables
⚡ Fast restart: vllm preload keeps weights in GPU memory across restarts, with data parallelism and MTP draft support; experimental engine snapshots
🛠️ Model Runner V2: draft-model speculative decoding, LiLiCorr, async DFlash, and DSpark adaptive verification for Gemma4
🌐 Large-scale serving: MoonEP balanced all-to-all, prefill context parallelism with data parallelism, DeepEPv2, and RL weight transfer
🎛️ Smarter scheduling: independent active-sequence limits, adaptive long-prefill chunking, and priority for requests already holding KV blocks
🗄️ HiSparse hardening: fixes for MTP verification, chunked-prefill livelocks, and host/GPU prefix caching
Thanks to everyone who made this release possible! 💙
Full release notes 👇