Running Kimi K3 on Hopper? We released an FP8-Block quantized checkpoint tuned for it. FP8 is native to Hopper's tensor cores, so you get the throughput the H100/H200 are built to deliver. Day Zero support ready with @vllm_project. Weights - try it now:

Lossless quantization has usually meant giving up inference speedup. This paper changes that. SLQ (Statistically-Lossless Quantization) reaches task-lossless compression at 3.3 bits per parameter, and distribution-lossless at 5-6 bpp where the output distribution is practically indistinguishable from the original. 1.7 to 3.6x throughput over BF16 in @vllm_project. Beats FP8 while staying lossless. From Michael Helcig, @_EldarKurtic, and @DAlistarh.

Ready to scale Models-as-a-Service across your organization without surprising cloud bills or shadow AI? Learn how binding self-service API keys to automated token controls gives platform teams granular visibility and security by default. Check out the blog post:

vLLM Office Hours today at 2pm ET: RL at 1T Scale, a prime-rl performance deep dive with @m_sirovatka (@PrimeIntellect). Training trillion-parameter MoE models like GLM-5.1 on agentic RL, plus what's new in vLLM 0.25 by @mgoin_. Get a recurring cal invite:

We just shipped DFlash speculator checkpoints for Qwen3.5-397B-A17B. On math_reasoning: ~5 out of 7 draft tokens accepted on average. On code (HumanEval): ~4.5 out of 7. Both checkpoints trained with the open source Speculators library from @vllm_project. Apache 2.0. Validated on NVIDIA H200. One flag to enable in vLLM: --speculative-config '{ "model": "RedHatAI/Qwen3.5-397B-A17B-speculator.dflash", "num_speculative_tokens": 7, "method": "dflash" }' Check it out:

🎉 Congrats to @poolsideai on Laguna S 2.1, a new open-weight model built for agentic coding and long-horizon work. 🧠 118B sparse MoE, only 8B active per token, up to 1M context, thinking + no-thinking modes, OpenMDW-1.1 🔁 Built to stay on task across long, multi-step runs: plan, call tools, check its work, recover, keep going 🖥️ The official NVFP4 quant runs locally on a single @NVIDIAAI DGX Spark This model is a scale up of the Laguna XS 2.1 architecture and vLLM runs it out of the box. Keep your existing Laguna serve setup, and the poolside_v1 tool-call and reasoning parsers already work.

Two powerful new models now available on Red Hat AI Inference on IBM Cloud. Two different strengths. ☑️ Granite 4.1 30B for enterprise assistants, RAG, code generation, and agentic workflows ☑️ Gemma 4 31B IT for multimodal reasoning, image understanding, and long-context applications Both are available via familiar OpenAI-compatible APIs. Nice work by the team expanding the model catalog and giving developers more choice without changing how they build and integrate AI applications. Getting started with these models on Red Hat AI Inference on IBM cloud:

Great writeup from @khluu000 👏 How does vLLM stay production-quality while merging ~2,000 commits/month and shipping every 2 weeks? The team broke down the three layers that make it possible. A huge community effort—thank you to everyone who is helping along the way.

Inkling running on a DGX B200 node with vLLM across 8 GPUs. Text, images, and reasoning from a single container. Red Hat AI FP8 checkpoints coming soon. Shoutout to the @vllm_project community for getting this up and running and @_soyr_ for the quick video.

Congrats to @Kimi_Moonshot on the Kimi K3 announcement! 🎉 Grateful for the shoutout and collab. The Kimi team announced they contributed a KDA prefix caching implementation directly to vLLM, to be released alongside the model. 🚀 KDA breaks assumptions behind conventional prefix caching, and this upstream work means the community gets efficient long-context serving from day 0. vLLM will support Kimi K3 on day-0 release. Open weights by July 27, 2026.

GLM 5.2 @ 19.8 tok/sec on 2 DGX Sparks ⚡️ I switched to a dspark draft model with K=2 (model from @RedHat_AI) Acceptance is ~68% because the drafter was trained with the full FP8 model... Next step is to fine tune the drafter so acceptance on this super quant goes up

If you've wanted to learn vLLM but don't have a GPU sitting around, this is the path. Free Red Hat Developer Sandbox account. JupyterLab and models already deployed. You connect, run the notebooks, and learn by doing. No setup required. In ~1 hour: quantize a model, serve it via an OpenAI-compatible API, and benchmark it under real load. The same end-to-end workflow you'd run in production, without the infrastructure overhead. Start here:

Red Hat AI just shipped DFlash speculator checkpoints for two of @NVIDIAAI's most powerful open models: → Nemotron Ultra 550B → Nemotron Super 120B On math and reasoning: ~5 out of 7 draft tokens accepted on average. On code (HumanEval): ~3.4 out of 7. Both checkpoints trained with the open source Speculators library from @vllm_project. Apache 2.0. Validated on NVIDIA B200. One flag to enable in vLLM: --spec-model RedHatAI/NVIDIA-Nemotron-3-Ultra-550B-A55B-speculator.dflash --spec-tokens 7 --spec-method dflash 🔗 Ultra 550B: 🔗 Super 120B:

To make agentic AI actually work at scale, we have to shift from just consuming tokens to providing them. Owning your inference infrastructure is the only way you're going to scale this cost-effectively without getting locked in. Watch more on @RedHat TV:

New free learning path on the Red Hat Developer Sandbox: compress, serve, and benchmark a model with @vllm_project, hands-on in Jupyter, no GPU needed. Here's what you'll actually do: ⚡ Quantize Qwen3 to W4A16 with LLM Compressor using GPTQ. Measure the result: 42% smaller, 8.2% perplexity increase. Learn to decide if that tradeoff fits your use case. 🚀 Connect to a running vLLM server and send requests via the OpenAI-compatible API. Watch 5 concurrent requests handled in real time. See prefix cache queries increment live via the Prometheus metrics endpoint. 📊 Run a GuideLLM benchmark: TTFT, inter-token latency, and E2E latency at p50, p95, and p99. Run Hellaswag with lm_eval. Cross-reference with the published model card to make a deployment decision backed by numbers. Less than an hour to complete. Free account. Built by @cedricclyburn and Michael Santos. 🙏

~5 minutes on what's new in @vllm_project v0.24 from core maintainer @mgoin_, including Blackwell RTX SparseMLA for DeepSeek V4 and GLM 5.2, dynamic speculative decoding, streaming parser engine, LibTorch stable ABI migration, and more.