Measuring a Spectrum, Not a Label

A video can look sharp and temporally coherent while still violating the dynamics of the physical world. PhyProbe treats physical consistency as a measurement problem: instead of only deciding whether something is wrong, it estimates how severe the violation is on a continuous scale anchored by real videos and clear failures.

How PhyProbe Works

PhyProbe uses a frozen 2B-parameter spatiotemporal visual encoder and trains only a lightweight 1M-parameter scoring head. Three complementary signals shape the score: pairwise rankings provide stable relative order, noisy human ratings provide approximate magnitude, and reference anchors map real videos near 0 and clear violations near 1. At inference time, the model needs only a single video—no prompt or reference pair.

Physical Video Benchmark Results

PhyProbe achieves the best weighted average and leads on three of four physical-consistency benchmarks. Its largest gains appear in real–generated and no-correspondence settings, while Gemini-3.1-Pro leads on VideoPhy2.

Pair types: R–G compares a real video with a generated video; G–G compares two generated videos.

Correspondence: image+text pairs share an input image and text prompt; text pairs share a prompt; no-correspondence pairs require neither.

Pairwise accuracy (%). Higher is better. Avg. is weighted by benchmark pair count; the best result in each column is highlighted.

Evaluation settings: ImplausiBench[1] (image+text, R–G) · VideoPhy2[2] (text, G–G) · PhyDetEx[3] (no correspondence, R–G) · VideoFeedback2[4] (no correspondence, G–G).

Alignment with Human Judgments

We compare PhyProbe with human ratings after aligning the score direction so that higher means more physically plausible. Correlation ranges from −1 to 1: values closer to 1 indicate stronger agreement, while values near 0 indicate little linear agreement (Pearson) or rank agreement (Spearman).

Spearman ρ

Do PhyProbe and humans rank videos in the same order?

Pearson r

Do PhyProbe's score values rise and fall with human rating values?

Takeaway: VideoPhy2-AutoEval is strongest on its in-domain VideoPhy2 benchmark, but its correlation falls substantially on VideoFeedback2. PhyProbe is moderate on VideoPhy2, strongest on VideoFeedback2, and achieves the best cross-dataset average. Average correlations use an equal-weight Fisher-z aggregation.

1. TRAVL: A Recipe for Making Video-Language Models Better Judges of Physics Implausibility
2. VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation
3. PhyDetEx: Detecting and Explaining the Physical Plausibility of T2V Models
4. VideoScore2: Think before You Score in Generative Video Evaluation

Transfer to Human Preference

Although trained without explicit general-preference labels, PhyProbe also transfers to broader video preference benchmarks: MonetBench[5], Rapidata-I2V[6][7], and VideoGen-RewardBench[8]. This suggests that physical failures are an important component of perceived video quality rather than an isolated artifact category.

Pairwise accuracy (%). Higher is better; the strongest result in each column is highlighted. For tie-aware evaluation, PhyProbe uses the 20th percentile of absolute score differences on each benchmark as its tie threshold. Without-tie results compare raw scores.

5. VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation
6. Rapidata: Image-to-Video Human Preference — Hailuo-02 vs. Marey
7. Rapidata: Image-to-Video Human Preference — Seedance-1-Pro
8. Improving Video Generation with Human Feedback
9. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning

Interpretable Violation Scores

In these examples, coherent real and generated motion receives scores near zero, while morphing, object pop-in, interpenetration, erratic motion, and impossible rigidity receive higher violation scores.