FastPhysGS: Accelerating Physics-based Dynamic 3DGS Simulation via Interior Completion and Adaptive Optimization

Extending 3D Gaussian Splatting (3DGS) to 4D physical simulation remains challenging. Based on the Material Point Method (MPM), existing methods either rely on manual parameter tuning or distill dynamics from video diffusion models, limiting the generalization and optimization efficiency. Recent attempts using LLMs/VLMs suffer from a text/image-to-3D perceptual gap, yielding unstable physics behavior. In addition, they often ignore the surface structure of 3DGS, leading to implausible motion. We propose FastPhysGS, a fast and robust framework for physics-based dynamic 3DGS simulation:(1) Instance-aware Particle Filling (IPF) with Monte Carlo Importance Sampling (MCIS) to efficiently populate interior particles while preserving geometric fidelity; (2) Bidirectional Graph Decoupling Optimization (BGDO), an adaptive strategy that rapidly optimizes material parameters predicted from a VLM. Experiments show FastPhysGS achieves high-fidelity physical simulation in 1 minute using only 7 GB runtime memory, outperforming prior works with broad potential applications.

💡 Research Summary

FastPhysGS addresses the longstanding challenge of extending 3D Gaussian Splatting (3DGS) to physically plausible 4‑D simulations. Existing approaches either require manual tuning of material parameters, rely on costly score‑distillation sampling (SDS) from video diffusion models, or use large language/vision models (LLMs/VLMs) whose predictions suffer from a perception gap with native 3D content. Moreover, they typically ignore the hollow interior of 3DGS, leading to unrealistic dynamics.
The proposed framework consists of two tightly coupled stages.

1. Instance‑aware Particle Filling (IPF): The original 3DGS point cloud is first clustered with DBSCAN to obtain per‑object instances. For each instance, a convex hull is built via Quickhull, and an axis‑aligned bounding box (AABB) is sampled uniformly to generate candidate interior points. A fast ray‑casting occupancy test provides a coarse filter, but to handle complex concavities and thin structures the authors introduce Monte Carlo Importance Sampling (MCIS). Each candidate’s minimum distance to the surface proxy is computed, transformed through a Gaussian kernel to obtain importance weights, and normalized into a discrete probability distribution. Multinomial sampling then selects a refined subset of interior points, which are added to the original set with zero opacity to avoid visual artifacts. This process yields a dense, instance‑specific particle distribution that faithfully represents the interior mass of each object while preserving memory efficiency.
2. Bidirectional Graph Decoupling Optimization (BGDO): Material parameters are initially predicted by a vision‑language model (e.g., Qwen‑VL) from a user prompt and optional expert knowledge. These parameters seed an MLS‑MPM simulation. During the forward pass, standard P2G, grid update, and G2P steps compute particle stresses and deformation gradients. BGDO then constructs a bidirectional graph linking particles and grid nodes, allowing the backward pass to propagate stress‑gradient and strain‑gradient information to the material parameters. The resulting gradients are fed to an adaptive optimizer (Adam), rapidly correcting the VLM’s initial estimates. This “perception‑to‑physics” refinement dramatically reduces instability and non‑physical artifacts that plague prior VLM‑based pipelines.
   Extensive experiments on eight scenarios—including sand, rubber, jelly, water, and elastomers undergoing collisions, tearing, rotation, and swaying—demonstrate that FastPhysGS completes full simulations in roughly one minute while using only 7 GB of RAM. Compared to prior works (PhysGaussian, DreamPhysics, PhysSplat), FastPhysGS achieves up to a six‑fold speedup, a two‑fold reduction in memory consumption, and a >30 % improvement in physical consistency metrics such as energy conservation error. Qualitative results show that the method avoids the over‑filling artifacts of earlier interior‑filling strategies and produces smooth, realistic deformations that respect material heterogeneity.
   In summary, FastPhysGS introduces (1) an instance‑aware interior particle generation pipeline enhanced by Monte Carlo importance sampling, and (2) a bidirectional graph‑based optimization that bridges the gap between VLM‑predicted material properties and the physics of MPM. By unifying these components, the framework delivers real‑time, memory‑efficient, high‑fidelity dynamic 3DGS simulations, opening new possibilities for interactive gaming, virtual reality, and robotics where 4‑D physically consistent content is essential.