Constrained Dynamic Gaussian Splatting

While Dynamic Gaussian Splatting enables high-fidelity 4D reconstruction, its deployment is severely hindered by a fundamental dilemma: unconstrained densification leads to excessive memory consumption incompatible with edge devices, whereas heuristic pruning fails to achieve optimal rendering quality under preset Gaussian budgets. In this work, we propose Constrained Dynamic Gaussian Splatting (CDGS), a novel framework that formulates dynamic scene reconstruction as a budget-constrained optimization problem to enforce a strict, user-defined Gaussian budget during training. Our key insight is to introduce a differentiable budget controller as the core optimization driver. Guided by a multi-modal unified importance score, this controller fuses geometric, motion, and perceptual cues for precise capacity regulation. To maximize the utility of this fixed budget, we further decouple the optimization of static and dynamic elements, employing an adaptive allocation mechanism that dynamically distributes capacity based on motion complexity. Furthermore, we implement a three-phase training strategy to seamlessly integrate these constraints, ensuring precise adherence to the target count. Coupled with a dual-mode hybrid compression scheme, CDGS not only strictly adheres to hardware constraints (error < 2%}) but also pushes the Pareto frontier of rate-distortion performance. Extensive experiments demonstrate that CDGS delivers optimal rendering quality under varying capacity limits, achieving over 3x compression compared to state-of-the-art methods.

💡 Research Summary

The paper addresses a critical bottleneck in deploying dynamic 3‑D Gaussian Splatting (3DGS) for free‑viewpoint video (FVV) on resource‑constrained devices. While recent dynamic extensions of 3DGS achieve impressive visual fidelity, they suffer from uncontrolled growth of Gaussian primitives, leading to excessive memory usage and rendering costs that exceed the capabilities of edge hardware. Existing solutions either prune after training or use heuristic growth schedules that do not guarantee strict adherence to a user‑specified budget, especially in dynamic scenes where motion information is essential for preserving visual quality.

To overcome these limitations, the authors propose Constrained Dynamic Gaussian Splatting (CDGS), a framework that formulates dynamic scene reconstruction as a budget‑constrained optimization problem. The central idea is to enforce a hard limit on the total number of Gaussians (N_target) during the entire training process, rather than treating it as a post‑hoc constraint. This is achieved through several novel components:

1. Differentiable Budget Controller – Each Gaussian is assigned a continuous activation variable c_i∈