Introducing SceneBound: How Much Should a Streaming 3D Reconstruction Model Remember?
SceneBound is a training-free rule that decides what a streaming 3D reconstruction model keeps in memory. Models such as CUT3R, StreamVGGT, STream3R and ZipMap read a video frame by frame and carry the past in a recurrent state, fast weights or a key-value (KV) cache, and every frame written there shifts the coordinates of later cameras or takes GPU memory. SceneBound stores a frame only when the camera reaches a new place: its predicted pose has to lie at least a radius M from every stored keyframe. Memory then grows with the places visited and stays flat when the camera returns. One radius serves four released models and three memory types.
Against the best published training-free rules we test:
• in one pass, 9% lower error in CUT3R's recurrent state
• in one pass, 18% lower error in StreamVGGT's KV cache
• after nine passes of a scene, 32% lower error with 11× less writing in the state
• after nine passes, 11% lower error with 2.2× less GPU memory in the cache
On 11 KITTI drives, its submaps cut the error of one state by 71%.
Paper:
Delighted to work with Chuyue
@artrix_fsc , Chutian, Jingmin
@KleinJoe219384 , Hongliang
@0xHongliang , Yuheng
@eiaton84855 and Xiuyu
@sheriyuo. The discussion with Lilo
@EpiphanyFall was also very helpful.