title: VolumeDiffusion: Flexible Text-to-3D Generation with Efficient
    Volumetric Encoder
summary: This paper introduces a pioneering 3D volumetric encoder designed for
  text-to-3D generation. To scale up the training data for the diffusion model, a
  lightweight network is developed to efficiently acquire feature volumes from
  multi-view images. The 3D volumes are then trained on a diffusion model for
  text-to-3D generation using a 3D U-Net. This research further addresses the
  challenges of inaccurate object captions and high-dimensional feature volumes.
  The proposed model, trained on the public Objaverse dataset, demonstrates
  promising outcomes in producing diverse and recognizable samples from text
  prompts. Notably, it empowers finer control over object part characteristics
  through textual cues, fostering model creativity by seamlessly combining
  multiple concepts within a single object. This research significantly
  contributes to the progress of 3D generation by introducing an efficient,
  flexible, and scalable representation methodology. Code is available at
  https://github.com/tzco/VolumeDiffusion.
paper_authors._id: 6581348660ccc7f2302a7679
paper_authors.name: Zhicong Tang
paper_authors._id: 6581348660ccc7f2302a767a
paper_authors.name: Shuyang Gu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-10T08:21:27.438Z
paper_authors._id: 64c38fcf573c5a427e12cd37
paper_authors.avatarUrl: /avatars/2b9de06f29147ed2c212e920afba0eaf.svg
paper_authors.fullname: cientgu
paper_authors.type: user
paper_authors.user: cientgu
paper_authors._id: 6581348660ccc7f2302a767b
paper_authors.name: Chunyu Wang
paper_authors._id: 6581348660ccc7f2302a767c
paper_authors.name: Ting Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T09:56:04.548Z
paper_authors._id: 6350ffa18445bbe32e926360
paper_authors.avatarUrl: /avatars/bc67d114191be7b16b463731a84c7e05.svg
paper_authors.fullname: Ting Zhang
paper_authors.type: user
paper_authors.user: hellozt
paper_authors._id: 6581348660ccc7f2302a767d
paper_authors.name: Jianmin Bao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T09:56:29.878Z
paper_authors._id: 6481e690f9ed842838a2b106
paper_authors.avatarUrl: /avatars/e89a3c8366df504a95dc08a1a412bf3d.svg
paper_authors.fullname: Jianmin Bao
paper_authors.type: user
paper_authors.user: jianmin-ustc
paper_authors._id: 6581348660ccc7f2302a767e
paper_authors.name: Dong Chen
paper_authors._id: 6581348660ccc7f2302a767f
paper_authors.name: Baining Guo
paper_title: VolumeDiffusion: Flexible Text-to-3D Generation with Efficient
    Volumetric Encoder
paper_summary: This paper introduces a pioneering 3D volumetric encoder designed for
  text-to-3D generation. To scale up the training data for the diffusion model, a
  lightweight network is developed to efficiently acquire feature volumes from
  multi-view images. The 3D volumes are then trained on a diffusion model for
  text-to-3D generation using a 3D U-Net. This research further addresses the
  challenges of inaccurate object captions and high-dimensional feature volumes.
  The proposed model, trained on the public Objaverse dataset, demonstrates
  promising outcomes in producing diverse and recognizable samples from text
  prompts. Notably, it empowers finer control over object part characteristics
  through textual cues, fostering model creativity by seamlessly combining
  multiple concepts within a single object. This research significantly
  contributes to the progress of 3D generation by introducing an efficient,
  flexible, and scalable representation methodology. Code is available at
  https://github.com/tzco/VolumeDiffusion.
paper_ai_keywords: 3D volumetric encoder diffusion model multi-view images 3D U-Net feature volumes Objaverse dataset text-to-3D generation object captions high-dimensional feature volumes automatic generation object part characteristics textual cues model creativity efficient representation flexible representation scalable representation