We just released TensorRT Model Connect in Public Preview. You can take a supported @huggingface model to end-to-end TensorRT inference in just two commands. No intermediate ONNX export, and the resulting bundle can run through native C++ APIs. We also built the entire project with @OpenAIDevs Codex agents, with humans directing and reviewing the work. That includes model implementations, performance tuning, tests, integrations, and docs. It’s open source, so go try it out, dig into the implementations, or contribute support for a new model:

Run #LLMs# on @NVIDIARobotics #Jetson# without x86 ONNX export. 🚀 #TensorRT-Model-Connect# on reComputer Classic J5012 (#AGXOrin# 64GB, #JP7#.2). @huggingface checkpoint → trtmc build → .bundle on device trtmc run for text gen Tested: #Qwen3-4B# FP16 (~41GB container peak, 64GB recommended) No ONNX hop. Edge LLM/agent ready. Would you use on-device TRTMC for prototyping, or keep ONNX export for production? 👇 Full guide: Get your own one: #NVIDIA# #Jetson# #TensorRT# #TRTMC# #AGXOrin# #EdgeAI# #LLM# #PhysicalAI# #SeeedStudio#

TL;DR FoundationStereo, the high-accuracy zero-shot stereo model, sped up by over 10x with almost no accuracy loss. It finally runs in real time. Title: Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching URL: Points 🎯 Problem: stereo had split into "accurate but heavy foundation models" vs "fast methods that sacrifice accuracy and generalization" 🧩 Divide-and-conquer: each of three bottlenecks accelerated separately (feature extraction, cost filtering, refinement) 📚 Feature extraction: distill hybrid monocular + stereo priors into a single efficient student 🔍 Cost filtering: blockwise NAS over 8 blocks cuts search from O(n^N) to O(n), with ILP minimizing error under a latency budget ✂️ Refinement: structured pruning via ConvGRU recurrent dependency graphs and Taylor-expansion importance 🗂 1.4M in-the-wild stereo pairs auto pseudo-labeled with a normal-consistency check ⚡ Middlebury-Q: 4.51% BP-2 at 49ms (vs 2.64% at 496ms, 10.1x slower), down to 21ms with TensorRT A new SOTA among real-time stereo methods, well suited for robotics and AR deployment. #StereoMatching# #ComputerVision#

3/ muse spark open is available today through API platforms at Baseten, Fireworks AI, Groq, Modal, Nebius, OpenRouter and Together AI. model weights are available for download on Amazon SageMaker AI, Google Cloud Model Garden, Hugging Face, and Microsoft Foundry Model Catalog. it runs on leading inference engines like SGLang, vLLM, TensorRT-LLM, Hugging Face Transformers, Apple MLX, llama.cpp, and Unsloth with optimizations for AMD, Apple, Intel, and NVIDIA silicon. developers can easily further customize the model using the PyTorch framework.

Shopee just published the results of building its own frontier LLM, which now powers use cases from search and personalization to supply chain. Compass, Shopee's specialized model for Southeast Asian ecommerce, scaled from 3 billion to 340 billion monthly API tokens in eight months on our AI factory stack. It includes NVIDIA Megatron-Core for pretraining, NVIDIA NeMo framework for post-training, and TensorRT-LLM for inference, running across NVIDIA A100, H100, and RTX PRO 6000 Blackwell GPUs. Results: ✅ Compass API tokens scale 113x in eight months ✅ 50x faster fraud detection at 90% lower processing cost Read the customer story.

We’re excited to welcome Mooncake to the PyTorch Ecosystem! Mooncake is designed to solve the “memory wall” in LLM serving. By integrating Mooncake’s high performance KVCache transfer and storage capabilities with PyTorch native inference engines like SGLang, vLLM, and TensorRT-LLM, it unlocks new levels of throughput and scalability for large language model deployments. Mooncake enables prefill decode disaggregation, global KVCache reuse, elastic expert parallelism, and serves as a fault tolerant PyTorch distributed backend. 🔗 #PyTorch# #OpenSourceAI# #LLM# #AIInfrastructure#

Sipeed is taking direct shots at DGX Spark, and their latest benchmarks show why TOPS alone is a bad Local AI buying metric. Sipeed's new chip is rated at just 64 INT8 TOPS. In its latest INT8 test it just beat DGX Spark ResNet50 🟢 AX8860 → 4,300 FPS 🟩 DGX Spark → 2,753 FPS YOLO26s 🟢 AX8860 → 1,450 FPS 🟩 DGX Spark → 617 FPS That's 56% faster on ResNet50 and 2.35× the throughput on YOLO26s. Sipeed's older 18-TOPS AX8850N manages 316 FPS on YOLO26s. DGX Spark → 617 FPS So an 18-TOPS edge NPU produced roughly half the FPS of Spark in this workload. Meanwhile, DGX Spark is marketed at up to 1,000 TOPS / 1 PFLOP FP4. ⚠️ These numbers are not directly comparable TOPS-for-TOPS. This benchmark is INT8. Nvidia's 1,000-TOPS is sparse FP4. Nvidia itself specifies that number as theoretical FP4 performance using sparsity. And these are Sipeed's own benchmarks, not independent testing. 🎯 The TOPS number on the box doesn't tell you how fast your workload will run. Precision, sparsity, kernels, architecture, memory bandwidth, and software utilization really matter. For Local AI, we would rather see model + quant + context + tps than a giant TOPS number. (which they have on the website) Sipeed says it used TensorRT INT8 on Spark with NVIDIA's recommended Docker environment and is inviting people to reproduce the results. Independent testing should be fun. 👀

This blog post from @nvidia is intended to develop a clearer way of reasoning about performance optimization for PyTorch models and other representations. It explores questions such as: What makes one compiled engine better than another? Why can individually promising decisions fail end to end? How should accuracy constrain performance optimization? It introduces the Global Performance Tuner, a feature for automatic performance tuning released in TensorRT 11.1, and explores these questions through an intuitive example and a mathematical framework. Read the full blog post here:

Topaz alternative 🤏 SeedVR2+TensorRT faster VAE Processing. VRGDG SeedVR2 TensorRT Studio turns the SeedVR2 pipeline into a practical Windows workflow: load a video, test preview, compare the result frame by frame, and complete long renders 👇