Physical information is sitting inside video embeddings, but models can't quite pull it out. This paper tests that gap with 8,000 controlled physics simulations.
Title: World Embedding Benchmark
URL:
Built from 80 families across fluid mechanics, solid mechanics, dynamics, and optics, this benchmark measures how much physics video embeddings actually "understand." Three highlights stand out.
🔍 Physical information exists, but is hard to extract
Pre-trained models show weak retrieval (Recall
@10 of just 0.6-7.3%), yet V-JEPA2 achieves strong physical-property regression (nRMSE ~0.94-1.02) despite having no text alignment at all. The physics is encoded — it's just not well connected to language.
⚖️ A tradeoff between alignment and recoverability
Physics-specific contrastive training improves retrieval and pair classification, but actually hurts physical-property regression. Reorganizing the representation space for similarity matching comes at the cost of preserving quantitative physical information.
🎬 A real payoff in retrieval-augmented generation
Using a physics-adapted model to retrieve reference videos lifted the average generation score from 0.60 to 0.66, with the biggest fidelity gains in optics and thermal domains.
The takeaway for world-model design: alignment and recoverability need to be evaluated together, not optimized one at a time.
#
WorldModels# #
VideoGeneration#