Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with
  Image-Based Surface Representation
Generating high-quality 3D objects from textual descriptions remains a
challenging problem due to computational cost, the scarcity of 3D data, and
complex 3D representations. We introduce Geometry Image Diffusion
(GIMDiffusion), a novel Text-to-3D model that utilizes geometry images to
efficiently represent 3D shapes using 2D images, thereby avoiding the need for
complex 3D-aware architectures. By integrating a Collaborative Control
mechanism, we exploit the rich 2D priors of existing Text-to-Image models such
as Stable Diffusion. This enables strong generalization even with limited 3D
training data (allowing us to use only high-quality training data) as well as
retaining compatibility with guidance techniques such as IPAdapter. In short,
GIMDiffusion enables the generation of 3D assets at speeds comparable to
current Text-to-Image models. The generated objects consist of semantically
meaningful, separate parts and include internal structures, enhancing both
usability and versatility.
66daa0964414997d108994e4
Slava Elizarov
admin_assigned
2024-09-06T09:46:38.828Z
639b56244bbcd176745a6e6d
https://cdn-avatars.huggingface.co/v1/production/uploads/639b56244bbcd176745a6e6d/a1DRanMtKUNJNcrt3ZmOg.jpeg
Slava Elizarov
user
SlavaElizarov
66daa0964414997d108994e5
Ciara Rowles
extracted_confirmed
2024-09-06T06:38:55.482Z
63357214eb6132ca653020e7
https://cdn-avatars.huggingface.co/v1/production/uploads/63357214eb6132ca653020e7/A_imoyFDx30wgi0guG_s8.png
Ciara
user
CiaraRowles
66daa0964414997d108994e6
Simon Donné
extracted_pending
2024-09-06T06:26:36.560Z
634d06167f1e86614c0e8e1e
/avatars/c1d119f42f35e0e2d7e1a655d193478b.svg
Simon Donné
user
simondonn
Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with
  Image-Based Surface Representation
Generating high-quality 3D objects from textual descriptions remains a
challenging problem due to computational cost, the scarcity of 3D data, and
complex 3D representations. We introduce Geometry Image Diffusion
(GIMDiffusion), a novel Text-to-3D model that utilizes geometry images to
efficiently represent 3D shapes using 2D images, thereby avoiding the need for
complex 3D-aware architectures. By integrating a Collaborative Control
mechanism, we exploit the rich 2D priors of existing Text-to-Image models such
as Stable Diffusion. This enables strong generalization even with limited 3D
training data (allowing us to use only high-quality training data) as well as
retaining compatibility with guidance techniques such as IPAdapter. In short,
GIMDiffusion enables the generation of 3D assets at speeds comparable to
current Text-to-Image models. The generated objects consist of semantically
meaningful, separate parts and include internal structures, enhancing both
usability and versatility.
Geometry Image Diffusion leverages 2D representation and collaborative control from Text-to-Image models to generate high-quality 3D objects efficiently.
Geometry Image Diffusion GIMDiffusion Collaborative Control geometry images 3D-aware architectures Text-to-Image models Stable Diffusion IPAdapter 3D assets semantically meaningful internal structures