ContactGaussian-WM: Learning Physics-Grounded World Model from Videos

Developing world models that understand complex physical interactions is essential for advancing robotic planning and simulation.However, existing methods often struggle to accurately model the environment under conditions of data scarcity and complex contact-rich dynamic motion.To address these challenges, we propose ContactGaussian-WM, a differentiable physics-grounded rigid-body world model capable of learning intricate physical laws directly from sparse and contact-rich video sequences.Our framework consists of two core components: (1) a unified Gaussian representation for both visual appearance and collision geometry, and (2) an end-to-end differentiable learning framework that differentiates through a closed-form physics engine to infer physical properties from sparse visual observations.Extensive simulations and real-world evaluations demonstrate that ContactGaussian-WM outperforms state-of-the-art methods in learning complex scenarios, exhibiting robust generalization capabilities.Furthermore, we showcase the practical utility of our framework in downstream applications, including data synthesis and real-time MPC.

💡 Research Summary

ContactGaussian‑WM introduces a differentiable, physics‑grounded rigid‑body world model that can be learned directly from sparse, contact‑rich video sequences. The core idea is to represent both visual appearance and collision geometry with a unified set of isotropic Gaussian primitives (center c, scale s, opacity α, and color f). By enforcing spherical shapes, the distance and overlap between Gaussians can be computed analytically, enabling a fully differentiable collision detector that replaces the non‑differentiable mesh‑based GJK/EPA pipelines used in conventional simulators.

The dynamics are driven by a closed‑form, complementarity‑free contact model (based on