title: Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with
    Image-Based Surface Representation
summary: Generating high-quality 3D objects from textual descriptions remains a
  challenging problem due to computational cost, the scarcity of 3D data, and
  complex 3D representations. We introduce Geometry Image Diffusion
  (GIMDiffusion), a novel Text-to-3D model that utilizes geometry images to
  efficiently represent 3D shapes using 2D images, thereby avoiding the need for
  complex 3D-aware architectures. By integrating a Collaborative Control
  mechanism, we exploit the rich 2D priors of existing Text-to-Image models such
  as Stable Diffusion. This enables strong generalization even with limited 3D
  training data (allowing us to use only high-quality training data) as well as
  retaining compatibility with guidance techniques such as IPAdapter. In short,
  GIMDiffusion enables the generation of 3D assets at speeds comparable to
  current Text-to-Image models. The generated objects consist of semantically
  meaningful, separate parts and include internal structures, enhancing both
  usability and versatility.
paper_authors._id: 66daa0964414997d108994e4
paper_authors.name: Slava Elizarov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-06T09:46:38.828Z
paper_authors._id: 639b56244bbcd176745a6e6d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/639b56244bbcd176745a6e6d/a1DRanMtKUNJNcrt3ZmOg.jpeg
paper_authors.fullname: Slava Elizarov
paper_authors.type: user
paper_authors.user: SlavaElizarov
paper_authors._id: 66daa0964414997d108994e5
paper_authors.name: Ciara Rowles
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-09-06T06:38:55.482Z
paper_authors._id: 63357214eb6132ca653020e7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63357214eb6132ca653020e7/A_imoyFDx30wgi0guG_s8.png
paper_authors.fullname: Ciara
paper_authors.type: user
paper_authors.user: CiaraRowles
paper_authors._id: 66daa0964414997d108994e6
paper_authors.name: Simon Donné
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-09-06T06:26:36.560Z
paper_authors._id: 634d06167f1e86614c0e8e1e
paper_authors.avatarUrl: /avatars/c1d119f42f35e0e2d7e1a655d193478b.svg
paper_authors.fullname: Simon Donné
paper_authors.type: user
paper_authors.user: simondonn
paper_title: Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with
    Image-Based Surface Representation
paper_summary: Generating high-quality 3D objects from textual descriptions remains a
  challenging problem due to computational cost, the scarcity of 3D data, and
  complex 3D representations. We introduce Geometry Image Diffusion
  (GIMDiffusion), a novel Text-to-3D model that utilizes geometry images to
  efficiently represent 3D shapes using 2D images, thereby avoiding the need for
  complex 3D-aware architectures. By integrating a Collaborative Control
  mechanism, we exploit the rich 2D priors of existing Text-to-Image models such
  as Stable Diffusion. This enables strong generalization even with limited 3D
  training data (allowing us to use only high-quality training data) as well as
  retaining compatibility with guidance techniques such as IPAdapter. In short,
  GIMDiffusion enables the generation of 3D assets at speeds comparable to
  current Text-to-Image models. The generated objects consist of semantically
  meaningful, separate parts and include internal structures, enhancing both
  usability and versatility.
paper_ai_summary: Geometry Image Diffusion leverages 2D representation and collaborative control from Text-to-Image models to generate high-quality 3D objects efficiently.
paper_ai_keywords: Geometry Image Diffusion GIMDiffusion Collaborative Control geometry images 3D-aware architectures Text-to-Image models Stable Diffusion IPAdapter 3D assets semantically meaningful internal structures