GaussianCube: Structuring Gaussian Splatting using Optimal Transport for
  3D Generative Modeling
3D Gaussian Splatting (GS) have achieved considerable improvement over Neural
Radiance Fields in terms of 3D fitting fidelity and rendering speed. However,
this unstructured representation with scattered Gaussians poses a significant
challenge for generative modeling. To address the problem, we introduce
GaussianCube, a structured GS representation that is both powerful and
efficient for generative modeling. We achieve this by first proposing a
modified densification-constrained GS fitting algorithm which can yield
high-quality fitting results using a fixed number of free Gaussians, and then
re-arranging the Gaussians into a predefined voxel grid via Optimal Transport.
The structured grid representation allows us to use standard 3D U-Net as our
backbone in diffusion generative modeling without elaborate designs. Extensive
experiments conducted on ShapeNet and OmniObject3D show that our model achieves
state-of-the-art generation results both qualitatively and quantitatively,
underscoring the potential of GaussianCube as a powerful and versatile 3D
representation.
660642d0b85b7b4ea5233a91
Bowen Zhang
660642d0b85b7b4ea5233a92
Yiji Cheng
660642d0b85b7b4ea5233a93
Jiaolong Yang
660642d0b85b7b4ea5233a94
Chunyu Wang
660642d0b85b7b4ea5233a95
Feng Zhao
660642d0b85b7b4ea5233a96
Yansong Tang
660642d0b85b7b4ea5233a97
Dong Chen
660642d0b85b7b4ea5233a98
Baining Guo
GaussianCube: Structuring Gaussian Splatting using Optimal Transport for
  3D Generative Modeling
3D Gaussian Splatting (GS) have achieved considerable improvement over Neural
Radiance Fields in terms of 3D fitting fidelity and rendering speed. However,
this unstructured representation with scattered Gaussians poses a significant
challenge for generative modeling. To address the problem, we introduce
GaussianCube, a structured GS representation that is both powerful and
efficient for generative modeling. We achieve this by first proposing a
modified densification-constrained GS fitting algorithm which can yield
high-quality fitting results using a fixed number of free Gaussians, and then
re-arranging the Gaussians into a predefined voxel grid via Optimal Transport.
The structured grid representation allows us to use standard 3D U-Net as our
backbone in diffusion generative modeling without elaborate designs. Extensive
experiments conducted on ShapeNet and OmniObject3D show that our model achieves
state-of-the-art generation results both qualitatively and quantitatively,
underscoring the potential of GaussianCube as a powerful and versatile 3D
representation.
GaussianCube, a structured Gaussian Splatting representation, enables efficient and high-quality generative modeling using diffusion models and U-Net architecture.
Gaussian Splatting Neural Radiance Fields GaussianCube densification-constrained GS fitting algorithm Optimal Transport 3D U-Net diffusion generative modeling ShapeNet OmniObject3D