TL;DR: A survey that treats video as an incomplete projection of a hidden world and organizes video AI research into four operations: Reading, Writing, Sharing, and Interacting.
Title: Reading, Writing, Sharing, and Interacting with the World through Video
URL:
Key points
・👁️ Reading: infer semantics, space, time, physics, and causality from video
・🎬 Writing: generate video from text, images, camera paths, and other conditions
・🔗 Sharing: unify understanding and generation via shared tokens and objectives
・🕹️ Interacting: closed-loop world models that predict state transitions under actions
・🎞️ Temporal coherence (neighboring frames agree) is not world consistency (compatible with an admissible world trajectory)
・🪜 An evidence ladder grades claims from plausible output up to decision utility
The shift in emphasis from visual fidelity to persistent state and physical and causal validity is a useful way to read the field.
#
WorldModels# #
VideoGeneration#