Motion Forcing: A Decoupled Framework for Robust Video Generation in Motion Dynamics

The ultimate goal of video generation is to satisfy a fundamental trilemma: achieving high visual quality, maintaining rigorous physical consistency, and enabling precise controllability. While recent models can maintain this balance in simple, isolated scenarios, we observe that this equilibrium is fragile and often breaks down as scene complexity increases (e.g., involving collisions or dense traffic). To address this, we introduce \textbf{Motion Forcing}, a framework designed to stabilize this trilemma even in complex generative tasks. Our key insight is to explicitly decouple physical reasoning from visual synthesis via a hierarchical \textbf{``Point-Shape-Appearance’’} paradigm. This approach decomposes generation into verifiable stages: modeling complex dynamics as sparse geometric anchors (\textbf{Point}), expanding them into dynamic depth maps that explicitly resolve 3D geometry (\textbf{Shape}), and finally rendering high-fidelity textures (\textbf{Appearance}). Furthermore, to foster robust physical understanding, we employ a \textbf{Masked Point Recovery} strategy. By randomly masking input anchors during training and enforcing the reconstruction of complete dynamic depth, the model is compelled to move beyond passive pattern matching and learn latent physical laws (e.g., inertia) to infer missing trajectories. Extensive experiments on autonomous driving benchmarks show that Motion Forcing significantly outperforms state-of-the-art baselines, maintaining trilemma stability across complex scenes. Evaluations on physics and robotics further confirm our framework’s generality.

💡 Research Summary

The paper tackles a fundamental challenge in video generation: simultaneously achieving high visual fidelity, strict physical consistency, and fine‑grained controllability. Existing diffusion‑based video generators excel at rendering realistic textures but often violate physical laws—such as inertia, collision dynamics, and object permanence—especially in complex scenes with multiple interacting agents. This limitation is attributed to the entanglement of dynamics and appearance in end‑to‑end models, which tend to prioritize pixel‑level losses over long‑term physical coherence.

To resolve this, the authors propose Motion Forcing, a decoupled generation framework built around a hierarchical “Point‑Shape‑Appearance” paradigm. The process is divided into three verifiable stages:

1. Point – Sparse control signals are abstracted as geometric anchors. Each object is represented by the center and radius of its maximum inscribed circle, encoding planar motion and an implicit depth cue. This representation is lightweight, easily scripted, and can be derived from user sketches, language instructions, or automatic keypoint detectors.

2. Shape – The Point representation is transformed into a sequence of dense depth maps. Depth serves as an explicit 3D structural prior that resolves occlusions, collision ordering, and relative motion. Camera motion is incorporated by warping the first‑frame depth according to the target extrinsics, providing a pixel‑aligned, spatially precise conditioning signal that avoids the pitfalls of low‑dimensional pose embeddings.

3. Appearance – Conditioned on the verified depth, a diffusion backbone (UNet or DiT) renders high‑resolution RGB frames. Because the geometry is already fixed, the appearance network can focus on texture, lighting, and material details without compromising physical plausibility.

A key training innovation is Masked Point Recovery. During training, a random subset of input points is masked, and the model is forced to reconstruct the full dynamic depth sequence. This compels the network to internalize physical dynamics—such as inertia, depth ordering, and object permanence—so that it can infer missing trajectories in 3D space. Consequently, the model learns active physical reasoning rather than passive pattern copying.

The framework is evaluated on several domains. For autonomous driving, the authors curate a challenging benchmark from Waymo, DrivingDojo, and YouTube videos, featuring dense traffic, abrupt maneuvers, and multi‑vehicle collisions. Motion Forcing outperforms state‑of‑the‑art controllable generators (MoFA‑Video, STANCE) and large foundation models (Seed‑Dance 2.0, WAN 2.6) on Fréchet Video Distance, LPIPS, and a custom physical consistency metric. Qualitatively, the generated videos preserve correct occlusion order, realistic collision outcomes, and smooth ego‑vehicle trajectories while maintaining photorealistic textures.

Generalization is demonstrated on the Physion physics simulator and the Jaco Play robotic manipulation dataset. In physics simulations, the model correctly predicts bounce directions and energy transfer even when some control points are hidden, confirming that the masked recovery objective successfully teaches the network underlying physical laws. In robotic manipulation, user‑specified directional arrows are mapped to point controls, enabling a simulated hand to push objects accurately according to the intended force direction, with realistic contact dynamics.

The paper’s contributions are threefold: (1) a decoupled “Point‑Shape‑Appearance” generation pipeline that structurally separates dynamics from appearance, (2) a Masked Point Recovery training strategy that enforces active physical reasoning, and (3) a unified, flexible control primitive that works across autonomous driving, physics simulation, and robot manipulation. By bridging the domain gap between sparse control cues and dense video through an explicit depth intermediate, Motion Forcing achieves a stable balance of the three desiderata—visual quality, physical consistency, and controllability—making it a promising foundation for future “world models” in safety‑critical and interactive AI systems.