Register and share your invite link to earn from video plays and referrals.

Red Hat AI
@RedHat_AI
Accelerating AI innovation with open platforms and community. The future of AI is open.
2.1K Following    12.9K Followers
How is @0LatHQ powering time-sensitive AI applications? By using Red Hat AI Factory with NVIDIA as the foundation for Zerogrid, its distributed inference network. Using Red Hat Advanced Cluster Management, they manage edge nodes powered by NVIDIA Blackwell GPUs across decentralized datacenters from a single control plane. Learn more: #KubeCon#+#CloudNativeCon#
Show more
Red Hat Enterprise Linux 10.3 and 9.9 are here! Strengthen your hybrid cloud foundation against AI-driven threats with sealed images, native post-quantum security, and self-managed AI troubleshooting. Explore the new releases: #KubeCon#+#CloudNativeCon#
Show more
vLLM EPD, now upgraded 🚀 DeepInfer demonstrated the value of EPD disaggregation. vLLM’s latest EPD improvements deliver lower TTFT and higher throughput per GPU, counting all encoder GPUs, integrated with vLLM Router, SMG, and llm-d! Read more here!👇
Show more
Red Hat AI Elevate, a virtual tech showcase on Dec 9. A look at Red Hat AI 3.6, the Red Hat AI Factory with @nvidia, and the built-in guardrails for running agents in production without blowing up governance or cost. Save your seat:
Show more
In @NVIDIAAIDev's OpenShell, your agent never holds the real secret. It only sees a placeholder, and the real key gets swapped in at request time, bound to the one endpoint it's allowed to hit. All of the agent's outbound traffic goes through a default-deny policy proxy, so even if it gets prompt-injected and tries to curl your creds off somewhere, it can't reach anything you didn't allow. You can even pick which programs get to make the call: GitHub CLI yes, bare curl no. It runs on OpenShift too, though that path is still experimental. @LegareKerrison's blog below has loads of great getting started resources - check it out!
Show more
Nvidia has an open source project for sandboxing and managing the secrets you pass to agents. Goal: Make enterprises comfortable running agents in production. I wrote a quick blog on gist of it...
Show more
come here me blab about microscaling formats in vllm at the end 🤠
You can now upgrade PyTorch without rebuilding vLLM. As of v0.33.0, vLLM ships LibTorch ABI Stable CUDA wheels, so you swap the torch wheel and import it, no vLLM rebuild to pick up a new PyTorch release. How @vllm_project, @RedHat, and @AIatMeta pulled it off:
Show more
New vLLM Office Hours on advanced model compression, with Kyle Sayers and @_soyr_. They cover: - LLM Compressor's sequential pipeline and data parallel schemes - Settings most users never change - What observers do and how they affect quantization error - Expert pruning with REAP and MoNE
Show more
@vllm_project @RedHat_AI 59 community-run sessions is a lot of free knowledge. Office hours like that make vLLM so much easier to learn.
The agentic era is here. Is your infrastructure ready for it? From zero trust operating systems to autonomous cluster management, follow along for Red Hat’s announcements live from #KubeCon#+#CloudNativeCon# and join us at booth #116# to learn more!
Show more
76 guest speakers. 24 organizations. 59 sessions and counting. vLLM Office Hours is a bi-weekly livestream with the people building @vllm_project: a project update, a guest talk, and open Q&A. Open to all. Get a calendar invite:
Show more
Love seeing what the community builds on vLLM-Omni 🤩 OpenDirector: a local short-film workspace — generate, edit, export — powered by vLLM-Omni with a wide range of models. Congrats @hsliuunc! 🎬 🎥 OpenDirector: ⚡ vLLM-Omni:
Show more
Join us in this livestream next Tuesday, learn how to shrink models to fit in your laptop and make them go brrr 💨 We will have @mervenoyann @maximelabonne @TadimetiAditya @bartowski1182 and @mgoin_ giving a primer and know-hows on quantization with llama.cpp and vLLM ⚡️
Show more
0
101
689
100
Forward to community
Hola Madrid! 🇪🇸 vLLM is coming to town — join us Nov 12 with @RedHat_AI and @NVIDIAAI for an evening of open-source inference: vLLM, vLLM-omni, LLM Compressor, and llm-d on Kubernetes. Register:
Show more
the strongest open model yet runs at 800+ tok/s/user on VR200 vLLM *on day 0* can’t wait to see how much faster we make it over the coming weeks 🔥
Getting benchmark numbers is easy. Getting the right ones is hard. KV cache offloading can cut TTFT from 60s to 10s and lift throughput up to 5.8x, but only when your working set and storage tier line up. Here's how we benchmarked it, caveats and all:
Show more
Shift your open source security from reactive to ready. Join our global webinar on November 12, 2026, to see how to evolve your security operating model from provenance to remediation. Register for the virtual event:
Show more
vLLM v0.33.0 is out! 🚀 927 commits · 389 contributors · 122 first-timers 🤖 New models: Reflection Beam-1.0, Step-5 Preview, Nemotron Parse 3.5, MiniCPM-OCR 1.0 🔥 PyTorch 2.14 — CUDA 13.0-only wheels, libtorch ABI-stable 🎲 Spec decoding: XPress refiner, FLy lossy verify (1.04–1.31x), DSpark 1373→3390 tok/s on GB300 ⚡ NVIDIA decode: 1.5–2x latency decode on GB300, FA4 sparse MLA, 39–48% lower TTFT 🚀 New AWS EFA image + shorter vllm/vllm name 🐧 ROCm 10.0 default, MonoKernel 2.32x P50 Full notes ➡️
Show more
Two optimized checkpoints for NVIDIA Nemotron 3.5 Super VL, a 120B multimodal MoE for long-running agents. Quantization and speculative decoding stack for close to 2x on Hopper, accuracy intact. FP8 (LLM Compressor): DSpark (Speculators):
Show more
More state-of-the-art MXFP4 ckpts ready for inference on AMD GPUs and @vllm_project available at: