From a single casual smartphone video, build a free-viewpoint "moving 3D human" — no studio, no camera rig required. Title: 4DAnyone: Create Anyone in 4D from a Casual Monocular Video URL: A framework that turns an uncalibrated monocular video into multi-view-consistent video, then into 4D Gaussian Splatting. Three highlights. 📦 Highlight 1: Compress reference context from O(N) to O(1) (RCP) 4D reconstruction needs dozens of views, but conditioning on generated views bloats the context and weakens guidance. Reference Context Packing packs references into fixed-length mixed-resolution context, keeping compute constant as views grow. 🔄 Highlight 2: Rotate view groups by noise level (TCR) Independently denoised view groups can't share info, so structure drifts. Target Context Routing cyclically shifts groups at high noise to propagate global structure, then fixes groups at low noise to stabilize detail. The switch at t/T=0.2 was optimal. 🎮 Highlight 3: Game-engine data for in-the-wild generalization 38k in-house multi-view videos (318 actors, 24 cameras) plus real light-stage and monocular data, trained via a 3-stage curriculum. It hits PSNR 24.33 on generation consistency, clearly beating prior methods. Photorealistic 4D avatars from phone video in ~40 min — free-viewpoint capture for everyone. #4DReconstruction# #GaussianSplatting#

When generating 4D worlds from video models, you never had to detour through RGB — a new method goes directly from latent space to 4D. Beyond Pixels: From Video Priors to 4D Worlds 🔍 Overview The conventional cascade — video generation → RGB decoding → 4D reconstruction — inherently loses geometric information at the RGB step and amplifies generator-specific artifacts. This work proposes "latent-to-4D generation," directly connecting final denoised video VAE latents to a 4D decoder without RGB decoding. 🛠 Problems Solved · Geometric information degrades during RGB decoding · Generator-specific noise propagates through RGB into 4D reconstruction · New generators require retraining the entire 4D module All three are bypassed by treating the VAE latent space as the direct interface to 4D synthesis. ⚙ Method Three components: · Alignment Module (𝒜ϕ): Trilinear resampling + 3D convolutions transform video latents into the 4D decoder's token space · L4AR Attention: Hierarchical frame-wise spatial alignment + global temporal attention across frames · 4D Decoder: Jointly predicts per-frame camera parameters (9D) and dense world-space point maps Only these three components are trained (rank-16 LoRA included). Video generators, VAE, and Transformers stay frozen, trained on ~1K annotated clips. 📊 Results DINO-F1 (text-to-4D): 57.01–57.09 (Ours) vs. 53.27–54.21 (cascade baseline). Image-to-4D: 61.60 vs. 55.79 — a 5.81-point improvement. Human evaluators preferred our geometry and completeness in 66.8–72.1% of comparisons. Robustness test at contamination ρ=0.6: point-map drift 0.0053 vs. 0.3827 baseline (71× more stable). 🌐 Generalization A single checkpoint works unchanged across Wan T2V at 14B and 1.3B, and Wan I2V — all sharing the same VAE. Motion, appearance, camera, and trajectory controls remain functional without any retraining. #4DGeneration# #VideoAI#

Reconstruct a moving, deforming object in 4D (3D + time) from a single-camera video — even under heavy occlusion and large non-rigid motion 🎥 Title: Lift4D: Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wild URL: 🎥 Overview A test-time optimization framework combining single-view 3D prediction with deformable 3D Gaussian Splatting. It fuses learned geometry/appearance priors with the video to handle severe occlusion and non-rigid motion. ❓ Problem it solves ・Directly predicting 4D is limited by scarce training data and generalizes poorly ・Initializing 3D and refining with video alone breaks down on in-the-wild scenes with large deformation and occlusion 💡 Methodology Three components: ・Causal Latent Conditioning: aligns an image-to-3D model temporally without retraining, coupling adjacent latents at the ODE level, with t₀ trading off consistency vs per-frame fidelity ・Deformable Gaussians: sparse control nodes with time-varying SE(3) transforms + linear blend skinning deform a canonical representation across frames ・Occlusion-aware refinement: compares only visible pixels, while a view-conditioned diffusion prior completes unobserved surfaces 📊 Results ・Compared against STAG4D, PAD3R, L4GM, DreamMesh4D, V2M4, BANMo ・On Pexels data, point-tracking error EPE 0.072 (vs 0.119–0.211 for competitors) ・~30 min per 32-frame video on a single H200 GPU #3DReconstruction# #GaussianSplatting#