title: AGG: Amortized Generative 3D Gaussians for Single Image to 3D
summary: Given the growing need for automatic 3D content creation pipelines, various
  3D representations have been studied to generate 3D objects from a single
  image. Due to its superior rendering efficiency, 3D Gaussian splatting-based
  models have recently excelled in both 3D reconstruction and generation. 3D
  Gaussian splatting approaches for image to 3D generation are often
  optimization-based, requiring many computationally expensive score-distillation
  steps. To overcome these challenges, we introduce an Amortized Generative 3D
  Gaussian framework (AGG) that instantly produces 3D Gaussians from a single
  image, eliminating the need for per-instance optimization. Utilizing an
  intermediate hybrid representation, AGG decomposes the generation of 3D
  Gaussian locations and other appearance attributes for joint optimization.
  Moreover, we propose a cascaded pipeline that first generates a coarse
  representation of the 3D data and later upsamples it with a 3D Gaussian
  super-resolution module. Our method is evaluated against existing
  optimization-based 3D Gaussian frameworks and sampling-based pipelines
  utilizing other 3D representations, where AGG showcases competitive generation
  abilities both qualitatively and quantitatively while being several orders of
  magnitude faster. Project page: https://ir1d.github.io/AGG/
paper_authors._id: 659ccde599aeb9b5de3a65e3
paper_authors.name: Dejia Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-09T09:30:01.342Z
paper_authors._id: 62f687a0c58915315c4ff75d
paper_authors.avatarUrl: /avatars/b657180c7666735062782edd4f6a69c9.svg
paper_authors.fullname: Dejia Xu
paper_authors.type: user
paper_authors.user: ir1d
paper_authors._id: 659ccde599aeb9b5de3a65e4
paper_authors.name: Ye Yuan
paper_authors._id: 659ccde599aeb9b5de3a65e5
paper_authors.name: Morteza Mardani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-09T09:30:30.967Z
paper_authors._id: 6350842a0f376d3c482bc04a
paper_authors.avatarUrl: /avatars/dca5f6b3e867ed28f80882ea2b32c2c8.svg
paper_authors.fullname: morteza mardani
paper_authors.type: user
paper_authors.user: mortezamardani
paper_authors._id: 659ccde599aeb9b5de3a65e6
paper_authors.name: Sifei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-09T09:30:58.222Z
paper_authors._id: 62fab69f8cd542e895bafd6e
paper_authors.avatarUrl: /avatars/c553bff4bd52b9a4f79e9c76fa22e27e.svg
paper_authors.fullname: Sifei Liu
paper_authors.type: user
paper_authors.user: zwrq
paper_authors._id: 659ccde599aeb9b5de3a65e7
paper_authors.name: Jiaming Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-09T09:31:05.712Z
paper_authors._id: 6312c2f4bbaa385279d2da1b
paper_authors.avatarUrl: /avatars/2bf162889fa726ed18cc205b3f28609e.svg
paper_authors.fullname: Jiaming Song
paper_authors.type: user
paper_authors.user: jiamings
paper_authors._id: 659ccde599aeb9b5de3a65e8
paper_authors.name: Zhangyang Wang
paper_authors._id: 659ccde599aeb9b5de3a65e9
paper_authors.name: Arash Vahdat
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-09T09:31:39.970Z
paper_authors._id: 62f6956cd3bdacb7eec02920
paper_authors.avatarUrl: /avatars/b22db0823311f866c00db2efc4b9f814.svg
paper_authors.fullname: Arash Vahdat
paper_authors.type: user
paper_authors.user: avahdat
paper_title: AGG: Amortized Generative 3D Gaussians for Single Image to 3D
paper_summary: Given the growing need for automatic 3D content creation pipelines, various
  3D representations have been studied to generate 3D objects from a single
  image. Due to its superior rendering efficiency, 3D Gaussian splatting-based
  models have recently excelled in both 3D reconstruction and generation. 3D
  Gaussian splatting approaches for image to 3D generation are often
  optimization-based, requiring many computationally expensive score-distillation
  steps. To overcome these challenges, we introduce an Amortized Generative 3D
  Gaussian framework (AGG) that instantly produces 3D Gaussians from a single
  image, eliminating the need for per-instance optimization. Utilizing an
  intermediate hybrid representation, AGG decomposes the generation of 3D
  Gaussian locations and other appearance attributes for joint optimization.
  Moreover, we propose a cascaded pipeline that first generates a coarse
  representation of the 3D data and later upsamples it with a 3D Gaussian
  super-resolution module. Our method is evaluated against existing
  optimization-based 3D Gaussian frameworks and sampling-based pipelines
  utilizing other 3D representations, where AGG showcases competitive generation
  abilities both qualitatively and quantitatively while being several orders of
  magnitude faster. Project page: https://ir1d.github.io/AGG/
paper_ai_keywords: 3D Gaussian splatting Amortized Generative 3D Gaussian framework (AGG) intermediate hybrid representation 3D Gaussian locations joint optimization cascaded pipeline coarse representation 3D Gaussian super-resolution module optimization-based 3D Gaussian frameworks sampling-based pipelines