When generating 4D worlds from video models, you never had to detour through RGB — a new method goes directly from latent space to 4D. Beyond Pixels: From Video Priors to 4D Worlds 🔍 Overview The conventional cascade — video generation → RGB decoding → 4D reconstruction — inherently loses geometric information at the RGB step and amplifies generator-specific artifacts. This work proposes "latent-to-4D generation," directly connecting final denoised video VAE latents to a 4D decoder without RGB decoding. 🛠 Problems Solved · Geometric information degrades during RGB decoding · Generator-specific noise propagates through RGB into 4D reconstruction · New generators require retraining the entire 4D module All three are bypassed by treating the VAE latent space as the direct interface to 4D synthesis. ⚙ Method Three components: · Alignment Module (𝒜ϕ): Trilinear resampling + 3D convolutions transform video latents into the 4D decoder's token space · L4AR Attention: Hierarchical frame-wise spatial alignment + global temporal attention across frames · 4D Decoder: Jointly predicts per-frame camera parameters (9D) and dense world-space point maps Only these three components are trained (rank-16 LoRA included). Video generators, VAE, and Transformers stay frozen, trained on ~1K annotated clips. 📊 Results DINO-F1 (text-to-4D): 57.01–57.09 (Ours) vs. 53.27–54.21 (cascade baseline). Image-to-4D: 61.60 vs. 55.79 — a 5.81-point improvement. Human evaluators preferred our geometry and completeness in 66.8–72.1% of comparisons. Robustness test at contamination ρ=0.6: point-map drift 0.0053 vs. 0.3827 baseline (71× more stable). 🌐 Generalization A single checkpoint works unchanged across Wan T2V at 14B and 1.3B, and Wan I2V — all sharing the same VAE. Motion, appearance, camera, and trajectory controls remain functional without any retraining. #4DGeneration# #VideoAI#