Video and world models are increasingly evaluated on their understanding of real-world physics. Physics-IQ (Motamed et al., 2026), later refined by Physics-IQ Verified, measures how closely generated videos reproduce real physical events using automated motion-based metrics.Building on this work, we investigate how reference-based evaluations relate to human perception. Using 198 real recordings across 66 physical scenarios, we evaluate 26 video and world models through two complementary questions: Blind, which video looks more realistic without seeing the original recording; and Referenced, which video more faithfully reproduces what actually happened.Across 758,940 human judgments and 126,490 pairwise comparisons, reference-conditioned human rankings correlate more strongly with Physics-IQ Verified scores (Spearman 0.81) than blind realism rankings (0.66).These findings highlight two distinct aspects of video generation: reproducing an observed physical event and generating a convincing, plausible continuation. A real recording captures one possible outcome, but not necessarily the only physically plausible one. This distinction raises broader questions for video generation evaluation and post-training: what should automated metrics and reward models optimize for, fidelity to a specific observed outcome or the broader space of physically plausible outcomes? The details cover the setup and the comparison with Physics-IQ, all videos and votes are on Hugging Face, and our blog post walks through examples.
Table 1. Overall standings, World Model Physics benchmark. Scores from a Bradley–Terry fit rescaled to Elo; intervals are 95% Wald intervals. Click any row to view sample outputs in Figure 1.
Figure 2. Overall Elo against the provider's list price, cost on a log scale. Teal-ringed models are the Pareto frontier, no model is both cheaper and better, joined by a step line, so picking a budget on the x-axis reads off the best model at or under it. Faded models are dominated; whiskers are the score 95% CI. Prices are hand-maintained snapshots of provider list prices; 5 models without a recorded price are not shown. Hover a model for exact numbers.
We use crowd intelligence to deliver real, diverse human feedback at scale for model evaluation and post-training.
