TL;DR: A survey that treats video as an incomplete projection of a hidden world and organizes video AI research into four operations: Reading, Writing, Sharing, and Interacting. Title: Reading, Writing, Sharing, and Interacting with the World through Video URL: Key points ・👁️ Reading: infer semantics, space, time, physics, and causality from video ・🎬 Writing: generate video from text, images, camera paths, and other conditions ・🔗 Sharing: unify understanding and generation via shared tokens and objectives ・🕹️ Interacting: closed-loop world models that predict state transitions under actions ・🎞️ Temporal coherence (neighboring frames agree) is not world consistency (compatible with an admissible world trajectory) ・🪜 An evidence ladder grades claims from plausible output up to decision utility The shift in emphasis from visual fidelity to persistent state and physical and causal validity is a useful way to read the field. #WorldModels# #VideoGeneration#

🚀 Introducing Hy-Nomas: An Open Source Free-Form Interactive Video World Model Key Features: * Free-form interaction with text prompts and camera trajectories * Long-horizon generation with world memory * Lightweight LoRA adaptation for specialized domains such as robotics * Optimized for Inference on Consumer GPUs Test it and drop feedback anytime！ 🔗 Website: 💻 GitHub: 🤗 Hugging Face: 🎥 Demo: #Hy##WorldModels# #VideoGeneration# #GenerativeAI#

🌍 TL;DR: A training-free way to re-explore a recorded video from new camera angles just dropped. It handles viewpoint changes, hole-filling, and recovering earlier appearances entirely through a frozen model's own attention. Title: World in World: Explore the World with World Models URL: Points 🎥 Four evidence sources (source video, target-view scene, rendered geometry, retrieved history) get converted into clean K/V fed straight into the frozen model 🧭 CGAR routes queries toward reliable source-video tokens using persistent point correspondence and camera geometry 🎚 EWA independently tunes each auxiliary channel's contribution using existing attention outputs, with zero extra network evaluations 🏆 Top VBench score of 85.192 and lowest rotation error (2.8326°) against 6 baselines including ReCamMaster 🔧 Same framework also powers bullet-time rendering, video stabilization, editing, and motion transfer Getting this much versatility with zero retraining feels like a win for evidence engineering over model adaptation. #WorldModels# #VideoGeneration#

A world model you steer with text and camera controls, with a design that even runs on a single RTX 4090 🎮 Hy-Nomas: Towards Free-form Interactive World Generation An 8B autoregressive video world model from a Tencent team. One model handles free-form text, continuous camera control, and multi-turn memory. Three highlights: 🧠 Action-aware causal attention Each segment's text is split into a description that later segments can see and actions confined to their own time window. The aim is to curb action-order errors and repeated actions across segments. ⚡ 4-step distillation and light inference Sampling drops from 100 to 4 function evaluations, combined with INT8 and sparse attention. A 100-second video (1280x704, 24 FPS) is generated at a peak of 20.53 GiB on an RTX 4090. 📊 Strong lead on text control In human evaluation of prompt following, it wins 75.0% and loses 8.3% against LingBot-World v2. Spatial memory, however, still trails Genie 3. A good read for both the design and evaluation of interactive world generation. #WorldModels# #VideoGeneration#

TL;DR: AgentGarten is a virtual-world framework that keeps simulator-grade faithfulness while matching video-model realism, and it taught agents hide-and-seek tool-use strategies in a handful of rounds that took conventional RL millions of episodes. Title: AgentGarten: Code Worlds for Evolving Agents URL: Points 🎮 Couples a scene program (state, rules) with a neural renderer (visuals) through structured conditions like depth and normal maps, decoupling physics from appearance 🔬 A new technique, "Adversarial Forcing," makes history prefilling differentiable via bitwise-exact replay with block-by-block SDPA attention, where standard approaches weren't differentiable at all ⚡ Achieves real-time rendering at 480×832 and 36.5fps on a single H100 🤖 A "playbook" mechanism lets agents carry round-by-round reflections forward, accumulating knowledge with no gradient updates 🏃 In hide-and-seek, shelter-building emerged by round 4 (vs. ~25M episodes for self-play RL) and ramp use by round 10 (vs. ~100M episodes) 📈 Other environments — herding, a quarry loader — also show clear gains within just 4 rounds What's striking is how much learning efficiency you gain just by architecturally separating state dynamics from visual generation. #WorldModels# #AgentLearning#

Physical information is sitting inside video embeddings, but models can't quite pull it out. This paper tests that gap with 8,000 controlled physics simulations. Title: World Embedding Benchmark URL: Built from 80 families across fluid mechanics, solid mechanics, dynamics, and optics, this benchmark measures how much physics video embeddings actually "understand." Three highlights stand out. 🔍 Physical information exists, but is hard to extract Pre-trained models show weak retrieval (Recall@10 of just 0.6-7.3%), yet V-JEPA2 achieves strong physical-property regression (nRMSE ~0.94-1.02) despite having no text alignment at all. The physics is encoded — it's just not well connected to language. ⚖️ A tradeoff between alignment and recoverability Physics-specific contrastive training improves retrieval and pair classification, but actually hurts physical-property regression. Reorganizing the representation space for similarity matching comes at the cost of preserving quantitative physical information. 🎬 A real payoff in retrieval-augmented generation Using a physics-adapted model to retrieve reference videos lifted the average generation score from 0.60 to 0.66, with the biggest fidelity gains in optics and thermal domains. The takeaway for world-model design: alignment and recoverability need to be evaluated together, not optimized one at a time. #WorldModels# #VideoGeneration#

Hide an object behind a wall in a video generation model, and it might come back as something else entirely. This paper tackles that object permanence failure head-on. Title: Training Object Permanence in World Models URL: 📝 Overview The paper trains video models on object permanence and solidity — cognitive abilities human infants develop by six months old. It builds WROP, a benchmark of 150 synthetic task generators producing 1.5M training samples, and PWM-WROP, a 16B-parameter model fine-tuned on it. ❓ Problem Solved Models like Sora let objects vanish behind occluders and reappear as different objects, or pass straight through solid barriers. These failures undermine higher-level reasoning about collisions and cause and effect. 💡 Method & Approach 150 Blender-generated tasks are organized into six occlusion and solidity families. Structural parameters like object count and trajectory vary systematically while surface parameters like color and lighting are randomized, preventing models from succeeding through memorization. PWM-WROP fine-tunes NVIDIA's Cosmos3-Nano on this data. 📊 Results In 361 blind human pairwise comparisons, PWM-WROP ranked first among true-continuation models (Elo 1679.5), 224 points ahead of the runner-up. It topped every model on static occlusion tasks, but still struggled on solidity tasks like collisions. 🌍 Use Cases The team released the training data, model weights, and PWM, a native-PyTorch training stack for AWS Trainium2 — laying groundwork for physically grounded world models. #WorldModels# #VideoGeneration#

An AI model just rediscovered Kepler's law from latent space, with zero knowledge of orbital mechanics. And the same core architecture spans molecular dynamics, cell prediction, and even generating cancer treatment hypotheses. Title: JEPA-Anything: Learning Predictive Models across Different Worlds URL: In one sentence: it decomposes a target's representation into K orthogonal subspaces, each with its own dedicated predictor — "Orthogonal Predictive Factorization" (OPF) — and applies this single mechanism across radically different domains, from vision and biology to clinical data, control, and molecular dynamics. 🔭 Highlight 1: Rediscovering a physical law from latent space Trained only on orbital motion data, its latent frequency modes recovered Kepler's law f=(2π)⁻¹a⁻³/². The fitted slope was -1.4991 against a theoretical -1.5, with R²=0.9999999. 🧬 Highlight 2: Generating and validating a cancer treatment hypothesis Factor analysis on liver cancer data proposed combining IL-18 and CD73 blockade, which then showed the strongest tumor cell killing in actual patient-derived organoids. ⚛️ Highlight 3: Consistently strong across molecular dynamics and cell prediction It achieved the lowest error across 100-step molecular simulations of water, quartz, paracetamol, and benzene, and also beat prior methods on single-cell perturbation prediction. It's striking that one core architecture spans such wildly different scientific domains this well. #WorldModels# #AIforScience#

🤖 Turns out robot dexterity scales with data too, and now there's hard evidence for it. Title: GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation URL: 🧩 Overview AgiBot Research Team built GE-Act 2.0, a world-action model that predicts future visual states to guide robot actions, training every component from scratch on manipulation data instead of inheriting a pretrained video generator. ⚙️ Problem it solves Prior systems reused reconstruction-optimized video generators that don't guarantee action-relevant information, and it was unclear how to pretrain visual generation and inverse dynamics separately before connecting them. 🛠 Methodology Three pieces make it work: a 64x-compressed control-oriented autoencoder (CoAE), a Single-Step Visual Planner that generates a full future frame in one forward pass, and KASO, which fixes the "validity gap" between predicted futures and real actions via top-k selection. 📊 Results Scaling training data from 300 to 30,000 hours lifted the G1-OP robot's success rate from 17.1% to 44.1%, a 27.0-point gain. Even G2-90D, with under 2% of the data, gained 17.7 points, showing real cross-embodiment transfer. Object, color, and position grounding all exceed 90% accuracy. #Robotics# #WorldModels#

🎮 Code agents thrive because code is executable and verifiable. This paper brings the same idea to game development to accelerate world model training. Title: Agentic Game Development as a Verifiable Trajectory Data Engine for Scaling World Models URL: 🧩 Overview The paper identifies a "verifiability bottleneck": fuzzy proxies like CLIP scores are noisy and gameable as reward signals. It proposes tapping game engines' collision, physics, and navmesh checks as a dense, trustworthy reward source instead. ❗ Problem it solves Spatial generation models (video, 3D, world models) lack reliable reward signals, so scaling compute doesn't reliably translate into real capability — what the authors call the "unverifiability tax." 🛠 Method RLHEV combines dense engine-verified rewards with sparse human acceptance judgments. AWoMo runs a Propose-Render-Verify-Repair-Review loop as a developer agent. UWDP unifies development traces into a single structured protocol for training data. 📊 Results On UnitySceneBench, full RLHEV beats the fuzzy-proxy baseline by +0.098. Cross-engine generalization (Unity to Godot) rises from 0.15 to 0.35. Downstream, it boosts D4RL Gym-MuJoCo by +48.43% and R2R navigation by +0.79%. #WorldModels# #ReinforcementLearning#