Reconstruct a moving, deforming object in 4D (3D + time) from a single-camera video — even under heavy occlusion and large non-rigid motion 🎥 Title: Lift4D: Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wild URL: 🎥 Overview A test-time optimization framework combining single-view 3D prediction with deformable 3D Gaussian Splatting. It fuses learned geometry/appearance priors with the video to handle severe occlusion and non-rigid motion. ❓ Problem it solves ・Directly predicting 4D is limited by scarce training data and generalizes poorly ・Initializing 3D and refining with video alone breaks down on in-the-wild scenes with large deformation and occlusion 💡 Methodology Three components: ・Causal Latent Conditioning: aligns an image-to-3D model temporally without retraining, coupling adjacent latents at the ODE level, with t₀ trading off consistency vs per-frame fidelity ・Deformable Gaussians: sparse control nodes with time-varying SE(3) transforms + linear blend skinning deform a canonical representation across frames ・Occlusion-aware refinement: compares only visible pixels, while a view-conditioned diffusion prior completes unobserved surfaces 📊 Results ・Compared against STAG4D, PAD3R, L4GM, DreamMesh4D, V2M4, BANMo ・On Pexels data, point-tracking error EPE 0.072 (vs 0.119–0.211 for competitors) ・~30 min per 32-frame video on a single H200 GPU #3DReconstruction# #GaussianSplatting#