Problem Formulation. We study data attribution for motion in the finetuning setting. Given a query video and a finetuning dataset, we assign each training clip a motion-aware influence score that quantifies how it contributes to the dynamics observed in the query. Our framework satisfies two key criteria: (i) predictivity, rankings correlate with observed changes when finetuning on the most influential subsets, and (ii) efficiency, scales to modern video generators without expensive Hessian inversion or per-data integration.

Problem Formulation. We study data attribution for motion in the finetuning setting. Let $\mathcal{D}_{\textnormal{ft}}$Finetuning dataset containing video-conditioning pairs = $\{(\mathbf{v}_{n}, \mathbf{c}_{n})\}_{n=1}^{N}$Set of N training video clips with their conditioning signals be the finetuning corpus with $N$Total number of training clips in the dataset training clips. Given a query video $(\hat{\mathbf{v}},\hat{\mathbf{c}})$Query video and its conditioning that we want to understand, we assign to each training clip $(\mathbf{v}_{n},\mathbf{c}_{n})$A specific training clip from the dataset a motion-aware influence score $$ I(\mathbf{v}_{n}, \hat{\mathbf{v}}; \boldsymbol{\theta}) $$ that explains how it contributes to the dynamics observed in $\hat{\mathbf{v}}$The query video we're analyzing.

Our framework satisfies two key criteria: (i) predictivity, rankings correlate with observed changes when finetuning on the most influential subsets, and (ii) efficiency, scales to modern video generators without expensive Hessian inversion or per-data integration.

1. Scalable Gradient-Based Attribution

We make attribution practical for billion-parameter models through several approximations:

• Inverse-Hessian Approximation: Use gradient similarity with identity preconditioner instead of computing exact inverse-Hessian-vector products.
• Common Randomness: Evaluate training and test gradients under the same (timestep, noise) pairs to reduce variance and stabilize rankings.
• Single-Sample Estimator: Fix a single timestep and shared noise draw for all train-test pairs, reducing compute from O(|dataset| · |timesteps| · cost) to O(|dataset| · cost).
• Fastfood Projection: Apply structured Johnson-Lindenstrauss projection to compress gradients, reducing storage from O(|dataset| · dim) to O(|dataset| · projected_dim), making it tractable for modern models.

1. Scalable Gradient-Based Attribution

We make attribution practical for billion-parameter models through several approximations:

• Inverse-Hessian Approximation: Use gradient similarity with identity preconditioner instead of computing exact inverse-Hessian-vector products.
• Common Randomness: Evaluate training and test gradients under the same $(t, \boldsymbol{\epsilon})$Diffusion timestep $t$ and noise vector $\boldsymbol{\epsilon}$ sampled from standard Gaussian pairs to reduce variance and stabilize rankings: $$ I_{\textnormal{diff}}^{1}(\mathbf{x}_n,\mathbf{x}_{\textnormal{test}}) =\frac{1}{|\mathcal{T}|} \sum_{t,\boldsymbol{\epsilon}\in\mathcal{T}} \frac{\nabla_{\boldsymbol{\theta}}\mathcal{L}_{\textnormal{diff}}(\boldsymbol{\theta};\,\mathbf{x}_{\textnormal{test}},t,\boldsymbol{\epsilon})} {\|\nabla_{\boldsymbol{\theta}}\mathcal{L}_{\textnormal{diff}}(\boldsymbol{\theta};\,\mathbf{x}_{\textnormal{test}},t,\boldsymbol{\epsilon})\|} ^{\!\top} \frac{\nabla_{\boldsymbol{\theta}}\mathcal{L}_{\textnormal{diff}}(\boldsymbol{\theta};\,\mathbf{x}_n,t,\boldsymbol{\epsilon})} {\|\nabla_{\boldsymbol{\theta}}\mathcal{L}_{\textnormal{diff}}(\boldsymbol{\theta};\,\mathbf{x}_n,t,\boldsymbol{\epsilon})\|} $$
• Single-Sample Estimator: Fix a single timestep $t_{\textnormal{fix}}$Fixed diffusion timestep used consistently across all examples and shared noise $\boldsymbol{\epsilon}_{\textnormal{fix}} \sim \mathcal{N}(0,\mathbf{I})$Single fixed noise sample shared to reduce variance for all train-test pairs, reducing compute from $\mathcal{O}(|\mathcal{D}| \cdot |\mathcal{T}| \cdot B)$Cost with multiple timesteps per example to $\mathcal{O}(|\mathcal{D}| \cdot B)$Cost with single timestep per example where $B$Computational cost of one forward and backward pass is the cost of a forward+backward pass.
• Fastfood Projection: Apply structured Johnson-Lindenstrauss projection to compress gradients. Let $$ \mathbf{P} \in \mathbb{R}^{D' \times D} \quad\text{with}\quad \mathbf{P} :=\frac{1}{\xi\sqrt{D'}}\; \mathbf{S} \mathbf{Q} \mathbf{G} \boldsymbol{\Pi} \mathbf{Q} \mathbf{B} $$ This reduces storage from $\mathcal{O}(|\mathcal{D}| \cdot D)$ to $\mathcal{O}(|\mathcal{D}| \cdot D')$.

The final influence score uses projected, normalized gradients: $$ I_{\textnormal{diff}}^{3}(\mathbf{x}_n,\mathbf{x}_{\textnormal{test}}) ~=~ \tilde{\mathbf{g}}\!\big(\boldsymbol{\theta};\,\mathbf{x}_{\textnormal{test}}\big)^{\!\top} \;\; \tilde{\mathbf{g}}\!\big(\boldsymbol{\theta};\,\mathbf{x}_n\big) $$

2. Motion Attribution via Motion Masking

To isolate temporal dynamics from static appearance, we introduce motion-weighted gradients:

• Motion Detection: Use AllTracker to extract optical flow and motion magnitudes in pixel space.
• Motion Weighting: Compute motion magnitude at each location, min-max normalize to [0,1], and bilinearly downsample to match latent space dimensions.
• Loss-Space Masking: Reweight per-location gradients by motion masks, emphasizing dynamic regions and de-emphasizing static backgrounds.
• Motion-Aware Influence: Compute influence scores using motion-weighted gradients, so rankings identify training clips that shape motion rather than appearance.

2. Motion Attribution via Motion Masking

To isolate temporal dynamics from static appearance, we introduce motion-weighted gradients:

• Motion Detection: Given a video $\mathbf{v} \in \mathbb{R}^{F \times H \times W \times 3}$Video with F frames, each of height H, width W, and 3 RGB channels with $F$Number of frames in the video frames of resolution $H \times W$Height and width of each frame in pixels, encode to latent space as $\mathbf{h} = E(\mathbf{v})$Latent representation via VAE encoder E $\in \mathbb{R}^{F \times H/s \times W/s \times C}$ with downsampling factor $s=8$Spatial downsampling factor (8× compression). Use AllTracker $\mathcal{A}$AllTracker motion detection model to extract motion: $A = \mathcal{A}(\mathbf{v})$Motion tensor with optical flow and confidence containing optical flow displacement vectors $\mathbf{D}_{f}(h,w) = (\mathrm{d}w, \mathrm{d}h)$2D displacement showing pixel movement at location (h,w) at each pixel location.
• Motion Weighting: Compute motion magnitude $M_{f}(h,w) = \|\mathbf{D}_{f}(h,w)\|_2$L2 norm (magnitude) of displacement vector and min-max normalize to $[0,1]$Normalized range where 1 = maximum motion: $$ \mathbf{W}(f,h,w) ~=~ \frac{M_{f}(h,w) - \min_{f',h',w'} M_{f'}(h',w')} {\max_{f',h',w'} M_{f'}(h',w') - \min_{f',h',w'} M_{f'}(h',w') + \zeta} $$ Bilinearly downsample to latent grid dimensions.
• Motion-Weighted Loss: Define per-location squared error and compute motion-weighted loss: $$ \mathcal{L}_{\textnormal{mot}}(\boldsymbol{\theta};\mathbf{v},\mathbf{c}) ~=~\frac{1}{F_{\mathbf{v}}} \operatorname{mean}_{f,\tilde{h},\tilde{w}} \left[\tilde{\mathbf{W}}_{\mathbf{v},\mathbf{c}}(f,\tilde{h},\tilde{w})\cdot \tilde{\mathcal{L}}_{\boldsymbol{\theta},\mathbf{v},\mathbf{c}}(f, \tilde{h},\tilde{w})\right] $$
• Motion-Aware Influence: Replace standard gradients with motion-weighted gradients $\mathbf{g}_{\textnormal{mot}} := \nabla_{\boldsymbol{\theta}}\mathcal{L}_{\textnormal{mot}}$Gradient of motion-weighted loss w.r.t. model parameters: $$ I_{\textnormal{mot}}\!(\mathbf{v}_n,\!\hat{\mathbf{v}}) \!=\! \tilde{\mathbf{g}}_{\textnormal{mot}}\!(\boldsymbol{\theta}\!,\!\hat{\mathbf{v}})^{\!\top}\! \tilde{\mathbf{g}}_{\textnormal{mot}}\!(\boldsymbol{\theta}\!,\!\mathbf{v}_n) $$ This isolates influence on temporal dynamics rather than static content.

3. Influential Subset Selection

Single-query: We select the highest-scoring examples based on motion-aware influence scores.

Multi-query aggregation: For multiple query motions, we adopt majority voting: a training sample receives a vote if its score exceeds a percentile threshold for that query. We rank samples by total votes and select the top examples to form the finetuning subset, emphasizing samples consistently influential across queries.

3. Influential Subset Selection

Single-query selection: We select the highest-scoring examples: $$\mathcal{S} = \{\mathbf{v}_n \mid \mathbf{v}_n \text{ in top examples by } I_{\textnormal{mot}}(\mathbf{v}_n, \hat{\mathbf{v}})\}$$ In practice, this is chosen as a percentile of the dataset size (e.g., top 1-10%).

Multi-query aggregation via majority voting: For $Q$Number of query videos with different target motions query motions $\{\hat{\mathbf{v}}_q\}_{q=1}^Q$Set of Q query videos representing different motion patterns, a training sample receives a vote if its score exceeds a percentile threshold $\tau$Percentile threshold (e.g., 90th percentile) for voting for that query. The consensus score is: $$ \operatorname{MajVote}_n = \sum_{q=1}^{Q}\mathbb{I}\!\big[\, I_{\textnormal{mot}}(\mathbf{v}_n,\hat{\mathbf{v}}_q) > \tau \,\big] $$
We rank all training samples by $\operatorname{MajVote}_n$Total votes received across all query videos and select the top examples to form: $$ \mathcal{S}_{\textnormal{vote}} = \bigl\{\mathbf{v}_n \mid \mathbf{v}_n \text{ in top examples by } \operatorname{MajVote} \bigr\} $$
This emphasizes samples consistently influential across multiple queries, without requiring cross-query calibration of raw scores.