TL;DR: A survey that treats video as an incomplete projection of a hidden world and organizes video AI research into four operations: Reading, Writing, Sharing, and Interacting.
Title: Reading, Writing, Sharing, and Interacting with the World through Video
URL:
Key points
ใป๐๏ธ Reading: infer semantics, space, time, physics, and causality from video
ใป๐ฌ Writing: generate video from text, images, camera paths, and other conditions
ใป๐ Sharing: unify understanding and generation via shared tokens and objectives
ใป๐น๏ธ Interacting: closed-loop world models that predict state transitions under actions
ใป๐๏ธ Temporal coherence (neighboring frames agree) is not world consistency (compatible with an admissible world trajectory)
ใป๐ช An evidence ladder grades claims from plausible output up to decision utility
The shift in emphasis from visual fidelity to persistent state and physical and causal validity is a useful way to read the field.
#
WorldModels# #
VideoGeneration#