We just released TensorRT Model Connect in Public Preview. You can take a supported @huggingface model to end-to-end TensorRT inference in just two commands. No intermediate ONNX export, and the resulting bundle can run through native C++ APIs. We also built the entire project with @OpenAIDevs Codex agents, with humans directing and reviewing the work. That includes model implementations, performance tuning, tests, integrations, and docs. It’s open source, so go try it out, dig into the implementations, or contribute support for a new model:

Run #LLMs# on @NVIDIARobotics #Jetson# without x86 ONNX export. 🚀 #TensorRT-Model-Connect# on reComputer Classic J5012 (#AGXOrin# 64GB, #JP7#.2). @huggingface checkpoint → trtmc build → .bundle on device trtmc run for text gen Tested: #Qwen3-4B# FP16 (~41GB container peak, 64GB recommended) No ONNX hop. Edge LLM/agent ready. Would you use on-device TRTMC for prototyping, or keep ONNX export for production? 👇 Full guide: Get your own one: #NVIDIA# #Jetson# #TensorRT# #TRTMC# #AGXOrin# #EdgeAI# #LLM# #PhysicalAI# #SeeedStudio#

TL;DR FoundationStereo, the high-accuracy zero-shot stereo model, sped up by over 10x with almost no accuracy loss. It finally runs in real time. Title: Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching URL: Points 🎯 Problem: stereo had split into "accurate but heavy foundation models" vs "fast methods that sacrifice accuracy and generalization" 🧩 Divide-and-conquer: each of three bottlenecks accelerated separately (feature extraction, cost filtering, refinement) 📚 Feature extraction: distill hybrid monocular + stereo priors into a single efficient student 🔍 Cost filtering: blockwise NAS over 8 blocks cuts search from O(n^N) to O(n), with ILP minimizing error under a latency budget ✂️ Refinement: structured pruning via ConvGRU recurrent dependency graphs and Taylor-expansion importance 🗂 1.4M in-the-wild stereo pairs auto pseudo-labeled with a normal-consistency check ⚡ Middlebury-Q: 4.51% BP-2 at 49ms (vs 2.64% at 496ms, 10.1x slower), down to 21ms with TensorRT A new SOTA among real-time stereo methods, well suited for robotics and AR deployment. #StereoMatching# #ComputerVision#

Shopee just published the results of building its own frontier LLM, which now powers use cases from search and personalization to supply chain. Compass, Shopee's specialized model for Southeast Asian ecommerce, scaled from 3 billion to 340 billion monthly API tokens in eight months on our AI factory stack. It includes NVIDIA Megatron-Core for pretraining, NVIDIA NeMo framework for post-training, and TensorRT-LLM for inference, running across NVIDIA A100, H100, and RTX PRO 6000 Blackwell GPUs. Results: ✅ Compass API tokens scale 113x in eight months ✅ 50x faster fraud detection at 90% lower processing cost Read the customer story.

We’re excited to welcome Mooncake to the PyTorch Ecosystem! Mooncake is designed to solve the “memory wall” in LLM serving. By integrating Mooncake’s high performance KVCache transfer and storage capabilities with PyTorch native inference engines like SGLang, vLLM, and TensorRT-LLM, it unlocks new levels of throughput and scalability for large language model deployments. Mooncake enables prefill decode disaggregation, global KVCache reuse, elastic expert parallelism, and serves as a fault tolerant PyTorch distributed backend. 🔗 #PyTorch# #OpenSourceAI# #LLM# #AIInfrastructure#

Topaz alternative 🤏 SeedVR2+TensorRT faster VAE Processing. VRGDG SeedVR2 TensorRT Studio turns the SeedVR2 pipeline into a practical Windows workflow: load a video, test preview, compare the result frame by frame, and complete long renders 👇

What if every decode step gave the next one a head start? Meet Guess-Verify-Refine — a new hardware-aware sparse-attention algorithm from NVIDIA Research. Built for TensorRT LLM on Blackwell, it reuses temporal patterns across decode steps for: → 1.88x faster Top-K attention → 9.3% better end-to-end latency in low-latency serving Dive into the paper:

If you already have 4x DGX Sparks, you might not missing out much waiting for the 512 GB M5 Ultra. Here's why: Memory capacity: both 512 GB unified. Peak memory bandwidth: 4x Sparks: ~1092 GB/s aggregate (TP=4) M5 Ultra: ~1,200 GB/s They're almost identical on what matters most for local LLMs - memory capacity and bandwidth. Where the DGX Sparks still pull ahead Prefill is generally stronger, concurrency scales more cleanly, and the entire CUDA ecosystem (vLLM, TensorRT-LLM, etc.) is simply more mature for multi-node inference right now. Video and image generation will also probably be noticeably faster on the sparks due to CUDA. CUDA is still king. The Mac is cleaner and more efficient as a single box. But if you already own the 4x DGX Sparks, you can continue sleeping well.

Open weights are table stakes now So here's the boring infra nobody wants to tweet about, also open⚡️ Things we've open sourced: — CubeSandbox, per-conversation sandboxes with browser, code, shell and file access, spun up and torn down for you — Agent Memory, a shared memory layer across your agents, and now across your teammates' agents too Things we've upstreamed: — RoCE optimizations for DeepSeek's DeepEP, doubling throughput on commodity networks, with 30% carrying back to InfiniBand — FlexKV, tiered KV cache offloading, now in NVIDIA Dynamo, vLLM and TensorRT-LLM mainlines All four are searchable by name, repos and docs. Pull requests welcome, and so are complaints 🛠️