A video generation model can produce plausible clips — but can it reproduce the correct distribution of futures? These are very different things. Title: PAWBench: How Far Are We from Probabilistically Aligned World Modeling? URL: 🎯 Overview Just as a coin should land heads roughly 50% of the time over many flips, a video generator should reproduce physically correct outcome frequencies — not just look realistic. PAWBench is the first systematic benchmark for this "probabilistic alignment," evaluating 50 scenarios across 11 video generation models. 🔍 The Problem Prior evaluations focus on perceptual quality (FID, FVD) or diversity, but miss whether models generate each outcome at the right frequency (calibration) and whether they can produce all possible outcomes at all (coverage). PAWBench fills this gap. ⚙️ Evaluation Method PAWEval generates K=50 video rollouts, maps them to terminal outcomes via Gemini 3.5 Flash, and scores them using total variation distance (TVD) for calibration and valid-support recovery rate for coverage. Eight physical mechanism groups are tested: tossing, rotation, collision, material transition, and more. 📊 Key Results The best model (Cosmos 3 Super I2V) achieves TVD of only 20.5 (ideal = 0). The average TVD across all models is 31.2 — far above the 8.33 expected from finite-sample chance effects alone. Not a single model simultaneously achieves accurate probabilities, broad coverage, and high scoring reliability. ⚡ What Interventions Reveal Three approaches were tested: prompt engineering, coupled noise sampling, and LoRA fine-tuning. Each improved only partial metrics. Models consistently underreact to physically causal interventions and overreact to non-causal visual or textual cues. For research applying video generation to robotics, autonomous driving, or physical simulation, probabilistic alignment is now an essential evaluation axis. #VideoGeneration# #WorldModels#