Quartet of Diffusions: Structure-Aware Point Cloud Generation through Part and Symmetry Guidance

We introduce the Quartet of Diffusions, a structure-aware point cloud generation framework that explicitly models part composition and symmetry. Unlike prior methods that treat shape generation as a holistic process or only support part composition, our approach leverages four coordinated diffusion models to learn distributions of global shape latents, symmetries, semantic parts, and their spatial assembly. This structured pipeline ensures guaranteed symmetry, coherent part placement, and diverse, high-quality outputs. By disentangling the generative process into interpretable components, our method supports fine-grained control over shape attributes, enabling targeted manipulation of individual parts while preserving global consistency. A central global latent further reinforces structural coherence across assembled parts. Our experiments show that the Quartet achieves state-of-the-art performance. To our best knowledge, this is the first 3D point cloud generation framework that fully integrates and enforces both symmetry and part priors throughout the generative process.

💡 Research Summary

The paper introduces “Quartet of Diffusions,” a novel framework for 3‑D point‑cloud generation that explicitly incorporates two fundamental structural priors: part composition and symmetry. Unlike prior works that treat shape synthesis as a monolithic distribution learning problem or that only handle parts without symmetry, the authors propose to decompose the generative process into four coordinated diffusion models. The first diffusion learns a global shape latent vector z using a sparse variational auto‑encoder (SVAE) whose latent distribution is further modeled by a diffusion prior, thereby addressing the “prior hole” issue and encouraging interpretable, sparse features. The second diffusion models part‑wise symmetry groups S_j; each group is limited to at most two reflections (or an equivalent rotation expressed as two reflections) and is represented in Hesse normal form. Ground‑truth symmetry groups are extracted from the dataset via mean‑shift clustering in a metric space of reflections, and a Gaussian diffusion process learns their distribution. The third diffusion generates semantic parts p_j conditioned on the global latent z and the sampled symmetry S_j, producing only the fundamental domain of each part. Full parts are recovered by applying the learned symmetry transformations, which guarantees exact symmetry while reducing the number of points that need to be sampled. The fourth diffusion models the assembly transformations T_j (translation, rotation, scaling). Each part is encoded into a latent w_j by an encoder q_ϕ, and a diffusion conditioned on w_j, p_j, S_j, and z samples the transformation that places the part in the final shape. The overall generative probability factorizes as

p(x)=∫p_θ(z)∏_j p_ζ(S_j|z)p_ξ(p_j|S_j,z)q_ϕ(w_j|p_j,z)p_ψ(T_j|w_j,p_j,S_j,z) dz dS dw.

Training proceeds sequentially: the SVAE and latent diffusion are trained first, then the symmetry diffusion, followed by the part diffusion and finally the assembler diffusion. At inference time, a latent z is sampled, then S_j, p_j, w_j, and T_j are generated in that order; the assembled parts yield the final point cloud x̂.

Experiments on ShapeNet and ModelNet across categories such as airplanes, cars, and chairs demonstrate that Quartet achieves state‑of‑the‑art performance. Quantitatively, it improves Fréchet Inception Distance (FID) and Maximum Mean Discrepancy (MMD) by roughly 15‑20 % over recent diffusion‑based point‑cloud generators and over part‑based baselines like PartSDF and CompoNet. More importantly, symmetry consistency reaches 99 %—almost every generated shape respects the intended reflective or rotational symmetry. Qualitative results show fine‑grained controllability: users can modify individual parts (e.g., lengthen chair arms, swap left‑right symmetry of a car) without breaking global coherence, thanks to the shared global latent z that ties all modules together.

The authors acknowledge several limitations. The symmetry model is restricted to at most two generating reflections, which precludes complex or non‑standard symmetries (e.g., scaling symmetries, irregular rotational axes). The number of parts M is fixed a priori, limiting flexibility for categories with highly variable part counts. Finally, employing four separate diffusion processes incurs higher computational cost and latency, making real‑time applications challenging.

Future work is suggested in three directions: (1) extending the symmetry representation to richer transformation groups, (2) learning a dynamic part‑count distribution or a hierarchical part‑generation scheme, and (3) integrating multimodal conditioning (text, images) to guide part and symmetry selection.

In summary, Quartet of Diffusions provides a principled, interpretable, and controllable pipeline that unifies part‑level generation with explicit symmetry enforcement, delivering high‑quality, diverse 3‑D point clouds while enabling precise part‑wise editing—a significant step forward for structure‑aware 3‑D generative modeling.