TL;DR FoundationStereo, the high-accuracy zero-shot stereo model, sped up by over 10x with almost no accuracy loss. It finally runs in real time.
Title: Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching
URL:
Points
๐ฏ Problem: stereo had split into "accurate but heavy foundation models" vs "fast methods that sacrifice accuracy and generalization"
๐งฉ Divide-and-conquer: each of three bottlenecks accelerated separately (feature extraction, cost filtering, refinement)
๐ Feature extraction: distill hybrid monocular + stereo priors into a single efficient student
๐ Cost filtering: blockwise NAS over 8 blocks cuts search from O(n^N) to O(n), with ILP minimizing error under a latency budget
โ๏ธ Refinement: structured pruning via ConvGRU recurrent dependency graphs and Taylor-expansion importance
๐ 1.4M in-the-wild stereo pairs auto pseudo-labeled with a normal-consistency check
โก Middlebury-Q: 4.51% BP-2 at 49ms (vs 2.64% at 496ms, 10.1x slower), down to 21ms with TensorRT
A new SOTA among real-time stereo methods, well suited for robotics and AR deployment.
#
StereoMatching# #
ComputerVision#