TL;DR: A survey that treats video as an incomplete projection of a hidden world and organizes video AI research into four operations: Reading, Writing, Sharing, and Interacting. Title: Reading, Writing, Sharing, and Interacting with the World through Video URL: Key points ・👁️ Reading: infer semantics, space, time, physics, and causality from video ・🎬 Writing: generate video from text, images, camera paths, and other conditions ・🔗 Sharing: unify understanding and generation via shared tokens and objectives ・🕹️ Interacting: closed-loop world models that predict state transitions under actions ・🎞️ Temporal coherence (neighboring frames agree) is not world consistency (compatible with an admissible world trajectory) ・🪜 An evidence ladder grades claims from plausible output up to decision utility The shift in emphasis from visual fidelity to persistent state and physical and causal validity is a useful way to read the field. #WorldModels# #VideoGeneration#

🚀 Introducing Hy-Nomas: An Open Source Free-Form Interactive Video World Model Key Features: * Free-form interaction with text prompts and camera trajectories * Long-horizon generation with world memory * Lightweight LoRA adaptation for specialized domains such as robotics * Optimized for Inference on Consumer GPUs Test it and drop feedback anytime！ 🔗 Website: 💻 GitHub: 🤗 Hugging Face: 🎥 Demo: #Hy##WorldModels# #VideoGeneration# #GenerativeAI#

🌍 TL;DR: A training-free way to re-explore a recorded video from new camera angles just dropped. It handles viewpoint changes, hole-filling, and recovering earlier appearances entirely through a frozen model's own attention. Title: World in World: Explore the World with World Models URL: Points 🎥 Four evidence sources (source video, target-view scene, rendered geometry, retrieved history) get converted into clean K/V fed straight into the frozen model 🧭 CGAR routes queries toward reliable source-video tokens using persistent point correspondence and camera geometry 🎚 EWA independently tunes each auxiliary channel's contribution using existing attention outputs, with zero extra network evaluations 🏆 Top VBench score of 85.192 and lowest rotation error (2.8326°) against 6 baselines including ReCamMaster 🔧 Same framework also powers bullet-time rendering, video stabilization, editing, and motion transfer Getting this much versatility with zero retraining feels like a win for evidence engineering over model adaptation. #WorldModels# #VideoGeneration#

A new approach to physical world models — reason about "how the world changes" in language before generating any pixels. Title: PhiZero: A World Model Built Around Physical Language ❓ What is "Physical Language"? 💡 A discrete token sequence that directly represents state-transition patterns in a video — separate from visual appearance. Because dynamics and appearance are disentangled, the state transitions encoded from one video can be transferred to a completely different visual style. PhiZero's "Reason-then-render" paradigm predicts this physical language first, then generates video from it. ❓ Why did previous physical world models produce physically inconsistent results? 💡 They relied on direct pixel-space prediction, burying dynamics implicitly inside the model. Reconciling visual realism with physical consistency was difficult — leading to failures like a tennis ball hitting a rubber duck that doesn't react. The reasoning process was also opaque by design. ❓ What performance numbers did it achieve? 💡 State-of-the-art results across all six benchmarks: ・Physics-IQ IQ-Score: 41.2 (beats previous SOTA of 39.5) ・WorldModelBench: highest in physical compliance, commonsense, and overall score ・YoCausal causal reasoning: aggregated rank 2.0 (best) ・Tokenizer: just 256 tokens achieve PSNR 28.9 — 1/175th of Wan2.2-5B VAE's token count ❓ What does this mean for robotics and autonomous driving? 💡 Physical Language is appearance-agnostic, so zero-shot sim-to-real and human-to-robot transfer become possible. The team demonstrated encoding LIBERO simulation dynamics and replaying them as realistic video, and directly transferring human motion patterns to robot embodiments — all without task-specific training. #WorldModel# #EmbodiedAI#

Seeing InternVideo-Next explicitly characterize its architecture as "Encoder–Predictor–Decoder (EPD)" and defining the Predictor as a "latent world model" confirms a clear convergence in the field. 📉 From our early work on Context AutoEncoder (CAE, — where we decoupled the Encoder, Predictor ("Regressor" in the paper), and Decoder — to Meta's I-JEPA/V-JEPA shifting entirely to latent prediction, and now this. It seems we are all validating the same core intuition: Understanding ≠ Pixel Reconstruction. 💡 Decoupling representation learning from high-frequency detail generation to build World Models in latent space is evidently the path forward. Glad to see our early intuition resonating with the latest SOTA. 🫡 #AI# #ComputerVision# #WorldModel# #JEPA# #CAE#

A world model you steer with text and camera controls, with a design that even runs on a single RTX 4090 🎮 Hy-Nomas: Towards Free-form Interactive World Generation An 8B autoregressive video world model from a Tencent team. One model handles free-form text, continuous camera control, and multi-turn memory. Three highlights: 🧠 Action-aware causal attention Each segment's text is split into a description that later segments can see and actions confined to their own time window. The aim is to curb action-order errors and repeated actions across segments. ⚡ 4-step distillation and light inference Sampling drops from 100 to 4 function evaluations, combined with INT8 and sparse attention. A 100-second video (1280x704, 24 FPS) is generated at a peak of 20.53 GiB on an RTX 4090. 📊 Strong lead on text control In human evaluation of prompt following, it wins 75.0% and loses 8.3% against LingBot-World v2. Spatial memory, however, still trails Genie 3. A good read for both the design and evaluation of interactive world generation. #WorldModels# #VideoGeneration#

TL;DR: AgentGarten is a virtual-world framework that keeps simulator-grade faithfulness while matching video-model realism, and it taught agents hide-and-seek tool-use strategies in a handful of rounds that took conventional RL millions of episodes. Title: AgentGarten: Code Worlds for Evolving Agents URL: Points 🎮 Couples a scene program (state, rules) with a neural renderer (visuals) through structured conditions like depth and normal maps, decoupling physics from appearance 🔬 A new technique, "Adversarial Forcing," makes history prefilling differentiable via bitwise-exact replay with block-by-block SDPA attention, where standard approaches weren't differentiable at all ⚡ Achieves real-time rendering at 480×832 and 36.5fps on a single H100 🤖 A "playbook" mechanism lets agents carry round-by-round reflections forward, accumulating knowledge with no gradient updates 🏃 In hide-and-seek, shelter-building emerged by round 4 (vs. ~25M episodes for self-play RL) and ramp use by round 10 (vs. ~100M episodes) 📈 Other environments — herding, a quarry loader — also show clear gains within just 4 rounds What's striking is how much learning efficiency you gain just by architecturally separating state dynamics from visual generation. #WorldModels# #AgentLearning#

Physical information is sitting inside video embeddings, but models can't quite pull it out. This paper tests that gap with 8,000 controlled physics simulations. Title: World Embedding Benchmark URL: Built from 80 families across fluid mechanics, solid mechanics, dynamics, and optics, this benchmark measures how much physics video embeddings actually "understand." Three highlights stand out. 🔍 Physical information exists, but is hard to extract Pre-trained models show weak retrieval (Recall@10 of just 0.6-7.3%), yet V-JEPA2 achieves strong physical-property regression (nRMSE ~0.94-1.02) despite having no text alignment at all. The physics is encoded — it's just not well connected to language. ⚖️ A tradeoff between alignment and recoverability Physics-specific contrastive training improves retrieval and pair classification, but actually hurts physical-property regression. Reorganizing the representation space for similarity matching comes at the cost of preserving quantitative physical information. 🎬 A real payoff in retrieval-augmented generation Using a physics-adapted model to retrieve reference videos lifted the average generation score from 0.60 to 0.66, with the biggest fidelity gains in optics and thermal domains. The takeaway for world-model design: alignment and recoverability need to be evaluated together, not optimized one at a time. #WorldModels# #VideoGeneration#

Hide an object behind a wall in a video generation model, and it might come back as something else entirely. This paper tackles that object permanence failure head-on. Title: Training Object Permanence in World Models URL: 📝 Overview The paper trains video models on object permanence and solidity — cognitive abilities human infants develop by six months old. It builds WROP, a benchmark of 150 synthetic task generators producing 1.5M training samples, and PWM-WROP, a 16B-parameter model fine-tuned on it. ❓ Problem Solved Models like Sora let objects vanish behind occluders and reappear as different objects, or pass straight through solid barriers. These failures undermine higher-level reasoning about collisions and cause and effect. 💡 Method & Approach 150 Blender-generated tasks are organized into six occlusion and solidity families. Structural parameters like object count and trajectory vary systematically while surface parameters like color and lighting are randomized, preventing models from succeeding through memorization. PWM-WROP fine-tunes NVIDIA's Cosmos3-Nano on this data. 📊 Results In 361 blind human pairwise comparisons, PWM-WROP ranked first among true-continuation models (Elo 1679.5), 224 points ahead of the runner-up. It topped every model on static occlusion tasks, but still struggled on solidity tasks like collisions. 🌍 Use Cases The team released the training data, model weights, and PWM, a native-PyTorch training stack for AWS Trainium2 — laying groundwork for physically grounded world models. #WorldModels# #VideoGeneration#

An AI model just rediscovered Kepler's law from latent space, with zero knowledge of orbital mechanics. And the same core architecture spans molecular dynamics, cell prediction, and even generating cancer treatment hypotheses. Title: JEPA-Anything: Learning Predictive Models across Different Worlds URL: In one sentence: it decomposes a target's representation into K orthogonal subspaces, each with its own dedicated predictor — "Orthogonal Predictive Factorization" (OPF) — and applies this single mechanism across radically different domains, from vision and biology to clinical data, control, and molecular dynamics. 🔭 Highlight 1: Rediscovering a physical law from latent space Trained only on orbital motion data, its latent frequency modes recovered Kepler's law f=(2π)⁻¹a⁻³/². The fitted slope was -1.4991 against a theoretical -1.5, with R²=0.9999999. 🧬 Highlight 2: Generating and validating a cancer treatment hypothesis Factor analysis on liver cancer data proposed combining IL-18 and CD73 blockade, which then showed the strongest tumor cell killing in actual patient-derived organoids. ⚛️ Highlight 3: Consistently strong across molecular dynamics and cell prediction It achieved the lowest error across 100-step molecular simulations of water, quartz, paracetamol, and benzene, and also beat prior methods on single-cell perturbation prediction. It's striking that one core architecture spans such wildly different scientific domains this well. #WorldModels# #AIforScience#