🌍 TL;DR: A training-free way to re-explore a recorded video from new camera angles just dropped. It handles viewpoint changes, hole-filling, and recovering earlier appearances entirely through a frozen model's own attention. Title: World in World: Explore the World with World Models URL: Points 🎥 Four evidence sources (source video, target-view scene, rendered geometry, retrieved history) get converted into clean K/V fed straight into the frozen model 🧭 CGAR routes queries toward reliable source-video tokens using persistent point correspondence and camera geometry 🎚 EWA independently tunes each auxiliary channel's contribution using existing attention outputs, with zero extra network evaluations 🏆 Top VBench score of 85.192 and lowest rotation error (2.8326°) against 6 baselines including ReCamMaster 🔧 Same framework also powers bullet-time rendering, video stabilization, editing, and motion transfer Getting this much versatility with zero retraining feels like a win for evidence engineering over model adaptation. #WorldModels# #VideoGeneration#

Hide an object behind a wall in a video generation model, and it might come back as something else entirely. This paper tackles that object permanence failure head-on. Title: Training Object Permanence in World Models URL: 📝 Overview The paper trains video models on object permanence and solidity — cognitive abilities human infants develop by six months old. It builds WROP, a benchmark of 150 synthetic task generators producing 1.5M training samples, and PWM-WROP, a 16B-parameter model fine-tuned on it. ❓ Problem Solved Models like Sora let objects vanish behind occluders and reappear as different objects, or pass straight through solid barriers. These failures undermine higher-level reasoning about collisions and cause and effect. 💡 Method & Approach 150 Blender-generated tasks are organized into six occlusion and solidity families. Structural parameters like object count and trajectory vary systematically while surface parameters like color and lighting are randomized, preventing models from succeeding through memorization. PWM-WROP fine-tunes NVIDIA's Cosmos3-Nano on this data. 📊 Results In 361 blind human pairwise comparisons, PWM-WROP ranked first among true-continuation models (Elo 1679.5), 224 points ahead of the runner-up. It topped every model on static occlusion tasks, but still struggled on solidity tasks like collisions. 🌍 Use Cases The team released the training data, model weights, and PWM, a native-PyTorch training stack for AWS Trainium2 — laying groundwork for physically grounded world models. #WorldModels# #VideoGeneration#

An AI model just rediscovered Kepler's law from latent space, with zero knowledge of orbital mechanics. And the same core architecture spans molecular dynamics, cell prediction, and even generating cancer treatment hypotheses. Title: JEPA-Anything: Learning Predictive Models across Different Worlds URL: In one sentence: it decomposes a target's representation into K orthogonal subspaces, each with its own dedicated predictor — "Orthogonal Predictive Factorization" (OPF) — and applies this single mechanism across radically different domains, from vision and biology to clinical data, control, and molecular dynamics. 🔭 Highlight 1: Rediscovering a physical law from latent space Trained only on orbital motion data, its latent frequency modes recovered Kepler's law f=(2π)⁻¹a⁻³/². The fitted slope was -1.4991 against a theoretical -1.5, with R²=0.9999999. 🧬 Highlight 2: Generating and validating a cancer treatment hypothesis Factor analysis on liver cancer data proposed combining IL-18 and CD73 blockade, which then showed the strongest tumor cell killing in actual patient-derived organoids. ⚛️ Highlight 3: Consistently strong across molecular dynamics and cell prediction It achieved the lowest error across 100-step molecular simulations of water, quartz, paracetamol, and benzene, and also beat prior methods on single-cell perturbation prediction. It's striking that one core architecture spans such wildly different scientific domains this well. #WorldModels# #AIforScience#

🤖 Turns out robot dexterity scales with data too, and now there's hard evidence for it. Title: GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation URL: 🧩 Overview AgiBot Research Team built GE-Act 2.0, a world-action model that predicts future visual states to guide robot actions, training every component from scratch on manipulation data instead of inheriting a pretrained video generator. ⚙️ Problem it solves Prior systems reused reconstruction-optimized video generators that don't guarantee action-relevant information, and it was unclear how to pretrain visual generation and inverse dynamics separately before connecting them. 🛠 Methodology Three pieces make it work: a 64x-compressed control-oriented autoencoder (CoAE), a Single-Step Visual Planner that generates a full future frame in one forward pass, and KASO, which fixes the "validity gap" between predicted futures and real actions via top-k selection. 📊 Results Scaling training data from 300 to 30,000 hours lifted the G1-OP robot's success rate from 17.1% to 44.1%, a 27.0-point gain. Even G2-90D, with under 2% of the data, gained 17.7 points, showing real cross-embodiment transfer. Object, color, and position grounding all exceed 90% accuracy. #Robotics# #WorldModels#

🎮 Code agents thrive because code is executable and verifiable. This paper brings the same idea to game development to accelerate world model training. Title: Agentic Game Development as a Verifiable Trajectory Data Engine for Scaling World Models URL: 🧩 Overview The paper identifies a "verifiability bottleneck": fuzzy proxies like CLIP scores are noisy and gameable as reward signals. It proposes tapping game engines' collision, physics, and navmesh checks as a dense, trustworthy reward source instead. ❗ Problem it solves Spatial generation models (video, 3D, world models) lack reliable reward signals, so scaling compute doesn't reliably translate into real capability — what the authors call the "unverifiability tax." 🛠 Method RLHEV combines dense engine-verified rewards with sparse human acceptance judgments. AWoMo runs a Propose-Render-Verify-Repair-Review loop as a developer agent. UWDP unifies development traces into a single structured protocol for training data. 📊 Results On UnitySceneBench, full RLHEV beats the fuzzy-proxy baseline by +0.098. Cross-engine generalization (Unity to Godot) rises from 0.15 to 0.35. Downstream, it boosts D4RL Gym-MuJoCo by +48.43% and R2R navigation by +0.79%. #WorldModels# #ReinforcementLearning#

A video generation model can produce plausible clips — but can it reproduce the correct distribution of futures? These are very different things. Title: PAWBench: How Far Are We from Probabilistically Aligned World Modeling? URL: 🎯 Overview Just as a coin should land heads roughly 50% of the time over many flips, a video generator should reproduce physically correct outcome frequencies — not just look realistic. PAWBench is the first systematic benchmark for this "probabilistic alignment," evaluating 50 scenarios across 11 video generation models. 🔍 The Problem Prior evaluations focus on perceptual quality (FID, FVD) or diversity, but miss whether models generate each outcome at the right frequency (calibration) and whether they can produce all possible outcomes at all (coverage). PAWBench fills this gap. ⚙️ Evaluation Method PAWEval generates K=50 video rollouts, maps them to terminal outcomes via Gemini 3.5 Flash, and scores them using total variation distance (TVD) for calibration and valid-support recovery rate for coverage. Eight physical mechanism groups are tested: tossing, rotation, collision, material transition, and more. 📊 Key Results The best model (Cosmos 3 Super I2V) achieves TVD of only 20.5 (ideal = 0). The average TVD across all models is 31.2 — far above the 8.33 expected from finite-sample chance effects alone. Not a single model simultaneously achieves accurate probabilities, broad coverage, and high scoring reliability. ⚡ What Interventions Reveal Three approaches were tested: prompt engineering, coupled noise sampling, and LoRA fine-tuning. Each improved only partial metrics. Models consistently underreact to physically causal interventions and overreact to non-causal visual or textual cues. For research applying video generation to robotics, autonomous driving, or physical simulation, probabilistic alignment is now an essential evaluation axis. #VideoGeneration# #WorldModels#

Game world models tend to generate pixels directly, but pose and geometry drift the longer you roll them out. Marionette simply gives up the parts that must be exact — to a renderer. Title: Marionette: Predicting World States, Rendering Geometry, Painting Appearance URL: A world model that delegates geometry, occlusion, and motion to a deterministic renderer and lets the neural net paint only appearance. Three highlights. 🎮 Highlight 1: Predict an explicit 276-D world state Instead of pixels, it predicts a 3D state of joints, root trajectory, and rotations via two-stage autoregression. Control is just overwriting an action token (like a player's button press). Forcing a wrong action shifts the pose by ~31% — proof the control truly bites. 📐 Highlight 2: A zero-parameter deterministic renderer Six closed-form operations turn state into pose-control video, so world consistency, occlusion order, and metric scale hold by construction. Long-horizon failures are fixed by rules (e.g. a terrain collider) without retraining appearance — cutting penetration by 66%. 🎨 Highlight 3: Video diffusion for appearance only A Wan2.2-Fun-5B diffusion paints photorealistic RGB on top of exact geometry. Even via predicted state, FVD is 831 (799 from recorded state, 975 for pixel-autoregressive) — no detectable fidelity loss. Exactness to deterministic compute, appearance to the neural net — a clean division of labor. #WorldModels# #GameAI#

How do we make robot policies robust to rare but high-impact failures? Video #World# #Models# (WMs) are rapidly becoming a powerful tool for robotics, enabling policy evaluation and improvement by "imagining" future outcomes. But there's a catch: these imagined futures are typically nominal samples, making it easy to overlook the rare yet safety-critical events that matter most. In our new paper, StressDream: Steering Video World Models for Robust Policy Evaluation and Improvement, we explore a simple but powerful idea: 💡 Instead of passively sampling futures, actively steer world model imaginations toward high-impact yet still plausible scenarios. StressDream optimizes the initial diffusion noise at inference time, allowing us to generate targeted stress-test scenarios without retraining the world model. This enables: - More robust policy evaluation by exposing failure modes that random sampling often misses. - Improved policy optimization by training against challenging but realistic imagined futures. As generative world models become a foundation for #Physical# #AI#, the ability to systematically probe their "long tail" of plausible futures will be increasingly important for building reliable and trustworthy autonomous systems. 📌 𝖯𝗋𝗈𝗃𝖾𝖼𝗍 𝖯𝖺𝗀𝖾: 📄 𝖯𝖺𝗉𝖾𝗋: Work led by Junwon Seo, with a great set of collaborators: Sushant Veer, Thomas Ran Tian, Wenhao Ding, Apoorva Sharma, Karen Leung, Edward Schmerling, Andrea Bajcsy. @NVIDIADRIVE @NVIDIAAI #Robotics# #WorldModels# #PhysicalAISafety# #AISafety# #AutonomousSystems# #RobotLearnin#

🧠 A transformer can always attend back to the past, so it has no structural incentive to compress history into a compact latent state. That means it tends to hoard information instead of summarizing it — and that hurts generalization. The fix proposed here is to add a light auxiliary objective to next-token prediction: also predict your own next latent. By learning to guess its next hidden state given the next token, the model grows a compact internal state with consistent transition rules — theoretically, a "belief state" that's a sufficient statistic for predicting the future. The transformer stays parallel while a lightweight dynamics model enforces temporal consistency — essentially co-training a transformer and an RNN in parallel. And the payoff is concrete. On a Manhattan-taxi world model the latent rank is ~3× more compact than GPT's; on reasoning and planning it looks ahead correctly instead of taking shortcuts (Countdown 54.8% vs ~39%). On a 1.3B language model it preserves quality while variable-length self-speculative decoding makes inference 3.3× faster (MTP/JTP cap out around 1.7×). Best of all, you only add a small MLP at training time — inference stays a single transformer. Next-Latent Prediction Transformers Learn Compact World Models #Transformer# #WorldModels#

🌍 "When can self-supervised learning recover the world's true structure?" This theory paper from LeCun and colleagues proves the answer is: only when the latent variables are Gaussian. Title: When Does LeJEPA Learn a World Model? URL: 💡 Overview The paper pins down when LeJEPA (JEPA + Gaussian regularization SIGReg + alignment) can recover the world's latent variables linearly, up to rotation, from nonlinear observations. The key condition: the latents are Gaussian and evolve under an OU process. ⚠️ The problem If a representation distorts the world's true degrees of freedom, reliable planning and compositional generalization break down. It was unclear when self-supervised learning provably recovers world structure. 🛠 Approach and core insight ・The optimal representation extracts the "slowest features" of the latent process, ordered by eigenvalue ・Via Hermite polynomials and Mehler's formula, cross-view correlation decays as ρ^d for degree-d nonlinearity ・So alignment penalizes every degree of nonlinearity, making the linear map the unique optimum ・With linear identifiability, planning in latent space yields the same optimal actions as the true world (directly usable for control) ・Conversely, demanding the optimum always be linear forces the latent distribution to be Gaussian (uniqueness) 📊 Results ・SIGReg and VICReg keep R² > 0.999 for linear recovery up to 1024 dimensions ・Sweeping the generalized-normal family, R² peaks sharply at α=2 (Gaussian) ・In pixel-based robot control, Gaussian OU pairs hit R²=0.95, while non-Gaussian real trajectories stay at R²≤0.5 ・Control cost tracks R² monotonically, and the Gaussian encoder is oracle-level #WorldModels# #SelfSupervisedLearning#