A new approach to physical world models — reason about "how the world changes" in language before generating any pixels. Title: PhiZero: A World Model Built Around Physical Language ❓ What is "Physical Language"? 💡 A discrete token sequence that directly represents state-transition patterns in a video — separate from visual appearance. Because dynamics and appearance are disentangled, the state transitions encoded from one video can be transferred to a completely different visual style. PhiZero's "Reason-then-render" paradigm predicts this physical language first, then generates video from it. ❓ Why did previous physical world models produce physically inconsistent results? 💡 They relied on direct pixel-space prediction, burying dynamics implicitly inside the model. Reconciling visual realism with physical consistency was difficult — leading to failures like a tennis ball hitting a rubber duck that doesn't react. The reasoning process was also opaque by design. ❓ What performance numbers did it achieve? 💡 State-of-the-art results across all six benchmarks: ・Physics-IQ IQ-Score: 41.2 (beats previous SOTA of 39.5) ・WorldModelBench: highest in physical compliance, commonsense, and overall score ・YoCausal causal reasoning: aggregated rank 2.0 (best) ・Tokenizer: just 256 tokens achieve PSNR 28.9 — 1/175th of Wan2.2-5B VAE's token count ❓ What does this mean for robotics and autonomous driving? 💡 Physical Language is appearance-agnostic, so zero-shot sim-to-real and human-to-robot transfer become possible. The team demonstrated encoding LIBERO simulation dynamics and replaying them as realistic video, and directly transferring human motion patterns to robot embodiments — all without task-specific training. #WorldModel# #EmbodiedAI#

🧠 A transformer can always attend back to the past, so it has no structural incentive to compress history into a compact latent state. That means it tends to hoard information instead of summarizing it — and that hurts generalization. The fix proposed here is to add a light auxiliary objective to next-token prediction: also predict your own next latent. By learning to guess its next hidden state given the next token, the model grows a compact internal state with consistent transition rules — theoretically, a "belief state" that's a sufficient statistic for predicting the future. The transformer stays parallel while a lightweight dynamics model enforces temporal consistency — essentially co-training a transformer and an RNN in parallel. And the payoff is concrete. On a Manhattan-taxi world model the latent rank is ~3× more compact than GPT's; on reasoning and planning it looks ahead correctly instead of taking shortcuts (Countdown 54.8% vs ~39%). On a 1.3B language model it preserves quality while variable-length self-speculative decoding makes inference 3.3× faster (MTP/JTP cap out around 1.7×). Best of all, you only add a small MLP at training time — inference stays a single transformer. Next-Latent Prediction Transformers Learn Compact World Models #Transformer# #WorldModels#

🌍 "When can self-supervised learning recover the world's true structure?" This theory paper from LeCun and colleagues proves the answer is: only when the latent variables are Gaussian. Title: When Does LeJEPA Learn a World Model? URL: 💡 Overview The paper pins down when LeJEPA (JEPA + Gaussian regularization SIGReg + alignment) can recover the world's latent variables linearly, up to rotation, from nonlinear observations. The key condition: the latents are Gaussian and evolve under an OU process. ⚠️ The problem If a representation distorts the world's true degrees of freedom, reliable planning and compositional generalization break down. It was unclear when self-supervised learning provably recovers world structure. 🛠 Approach and core insight ・The optimal representation extracts the "slowest features" of the latent process, ordered by eigenvalue ・Via Hermite polynomials and Mehler's formula, cross-view correlation decays as ρ^d for degree-d nonlinearity ・So alignment penalizes every degree of nonlinearity, making the linear map the unique optimum ・With linear identifiability, planning in latent space yields the same optimal actions as the true world (directly usable for control) ・Conversely, demanding the optimum always be linear forces the latent distribution to be Gaussian (uniqueness) 📊 Results ・SIGReg and VICReg keep R² > 0.999 for linear recovery up to 1024 dimensions ・Sweeping the generalized-normal family, R² peaks sharply at α=2 (Gaussian) ・In pixel-based robot control, Gaussian OU pairs hit R²=0.95, while non-Gaussian real trajectories stay at R²≤0.5 ・Control cost tracks R² monotonically, and the Gaussian encoder is oracle-level #WorldModels# #SelfSupervisedLearning#

Interactive video world models that generate footage as you control them — here's a unified benchmark that finally measures them fairly 🎮 Title: WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation URL: 🎮 Overview WBench is a unified framework for comprehensively evaluating interactive video world models. With 289 test cases and 1,058 interaction turns, it unifies text, 6-DoF pose, and discrete-action control so models with different native inputs can be compared on equal footing. ❓ Challenges Solved Interactive world models are advancing fast, but there was no comprehensive standard to assess them. Existing benchmarks only partially covered the needed competencies, and differing input interfaces made apples-to-apples comparison hard. 💡 Methodology & Proposed Approach Evaluation spans five core dimensions. ・Video quality ・Setting adherence ・Interaction adherence ・Consistency ・Physics compliance Tasks cover navigation, subject action, event editing, and perspective switching. It uses 22 automatic sub-metrics combining specialist vision models with large multimodal models, all validated against human judgments. 📊 Experimental Results Analyzing 20 state-of-the-art models revealed that no single model performs strongly across all dimensions, exposing characteristic strengths, weaknesses, and persistent challenges across approaches. #WorldModels# #Benchmark#

🕶️ Walk through a first-person world with your own body motion, and explicitly specify what exists at a given location with an image and pose, including how it evolves over time. Meet AnchorWorld, an embodied egocentric world model. Title: AnchorWorld: Embodied Egocentric World Simulation with View-based Evolution Customization URL: 📝 Overview AnchorWorld generates first-person video controlled by full-body human motion. With "anchor views," it lets you explicitly specify what exists at a given 3D location and how it changes over time. ❓ Challenges Solved Existing world models struggle to supervise full-body motion from egocentric video alone, and define environments only implicitly. They lacked both natural embodied control and localized world customization. 💡 Methodology & Proposed Approach ・Since most of the body is invisible in first person, it uses third-person video as auxiliary supervision to learn body-environment positioning ・An anchor has three parts: an RGB image, a 6-DoF viewpoint pose, and an evolution prompt that specify local appearance and temporal change ・3D RoPE spatially distinguishes multiple anchors, and masked cross-attention enables anchor-specific text control ・It trains in four stages (third-person, first-person, static anchors, dynamic evolution), built on Wan 2.2 TI2V 5B 🎯 Use Cases It applies to embodied VR apps, first-person game environment design, embodied-AI training scenarios, and interactive video generation with localized control. 📊 Experimental Results ・On egocentric static scenes it reaches CLIP-V 0.885 and camera accuracy ATE 0.112m, beating PlayerOne and others ・On egocentric dynamic scenes, text alignment (VideoAlign-TA) is 0.717, far above CaM-Ego's 0.385 ・It generalizes strongly to out-of-distribution UE and real-world scenes with little visual overlap between the initial view and anchors #WorldModel# #EmbodiedAI#

Frontier world models now run on edge GPUs. ⚡ Now openly available, NVIDIA Cosmos 3 Edge brings advanced world-model capabilities to local devices. The 4-billion-parameter omnimodel can understand and generate text, image, video, ambient sound and action for physical AI across robotics, autonomous vehicles and smart infrastructure. 🔗 #SIGGRAPH2026#

I got to "play" a world model in real life. The @GoogleDeepMind folks set up a crazy demo for Genie. You select glowing orbs to represent your scene and character. It loads the world in the model, and you navigate with joysticks like a video game 🕹️

3rd Place: GrokWorld uses Grok Imagine as a world model to generate synthetic training data for robots — augmenting or replacing months of manual collection in hours. @apturaai