World Models' Last Exam in Physics

Headlines · 1

1. 3d
   New benchmark tests physical consistency of video world models; best scores 57.76/100

   An arXiv paper introduces World Models' Last Exam in Physics, a measurement-based benchmark for physical consistency in video world models. It covers 40 controlled tasks spanning mechanics, optics, fluids, thermal and phase-change phenomena, electromagnetism and surface tension, each pairing an initial image and generation prompt with predefined physical criteria; the evaluator combines task-observability screening with task-specific quantitative measurements. Across eight video generation models and 1,280 videos, physical inconsistencies persisted with wide variation between tasks, and the best model scored 57.76 out of 100. The authors report that on synthetic videos with known physical relationships, the evaluator agreed with human judgments more than a direct vision-language-model baseline in both within-task rankings and pairwise comparisons.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my World Models' Last Exam in Physics experienceAsk about World Models' Last Exam in Physics