TL;DR: By generating perspective and bird's-eye-view (BEV) video together, a driving world model can make surrounding vehicles react to the ego vehicle's maneuvers.
BirdDreams: Joint Perspective and BEV Video Generation for a Plausible, Reactive Driving World Model
Key points
・🚗 Prior models took surrounding-vehicle trajectories as external input, fixing them to logged motion
・🗺️ It generates perspective and BEV video jointly, so scene layout becomes part of the prediction rather than an input
・🧠 Two video diffusion transformers use joint attention at every block (about 4B parameters)
・🎲 Independent per-stream noise lets each branch learn to use the other's information
・📉 With ego-action conditioning, ADE drops about 35-36% vs OmniDreams (Plücker)
・👥 In a human study (25 participants), BirdDreams was preferred 57% vs 24% for OmniDreams
Caveats: ego trajectories are reconstructed from generated video, and the human study compares only variants given the same road geometry.
#
WorldModel# #
AutonomousDriving#