TL;DR FoundationStereo, the high-accuracy zero-shot stereo model, sped up by over 10x with almost no accuracy loss. It finally runs in real time.
Title: Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching
URL:
Points
🎯 Problem: stereo had split into "accurate but heavy foundation models" vs "fast methods that sacrifice accuracy and generalization"
🧩 Divide-and-conquer: each of three bottlenecks accelerated separately (feature extraction, cost filtering, refinement)
📚 Feature extraction: distill hybrid monocular + stereo priors into a single efficient student
🔍 Cost filtering: blockwise NAS over 8 blocks cuts search from O(n^N) to O(n), with ILP minimizing error under a latency budget
✂️ Refinement: structured pruning via ConvGRU recurrent dependency graphs and Taylor-expansion importance
🗂 1.4M in-the-wild stereo pairs auto pseudo-labeled with a normal-consistency check
⚡ Middlebury-Q: 4.51% BP-2 at 49ms (vs 2.64% at 496ms, 10.1x slower), down to 21ms with TensorRT
A new SOTA among real-time stereo methods, well suited for robotics and AR deployment.
#
StereoMatching# #
ComputerVision#