VolumeDiffusion: Flexible Text-to-3D Generation with Efficient
  Volumetric Encoder
This paper introduces a pioneering 3D volumetric encoder designed for
text-to-3D generation. To scale up the training data for the diffusion model, a
lightweight network is developed to efficiently acquire feature volumes from
multi-view images. The 3D volumes are then trained on a diffusion model for
text-to-3D generation using a 3D U-Net. This research further addresses the
challenges of inaccurate object captions and high-dimensional feature volumes.
The proposed model, trained on the public Objaverse dataset, demonstrates
promising outcomes in producing diverse and recognizable samples from text
prompts. Notably, it empowers finer control over object part characteristics
through textual cues, fostering model creativity by seamlessly combining
multiple concepts within a single object. This research significantly
contributes to the progress of 3D generation by introducing an efficient,
flexible, and scalable representation methodology. Code is available at
https://github.com/tzco/VolumeDiffusion.
6581348660ccc7f2302a7679
Zhicong Tang
6581348660ccc7f2302a767a
Shuyang Gu
claimed_verified
2024-07-10T08:21:27.438Z
64c38fcf573c5a427e12cd37
/avatars/2b9de06f29147ed2c212e920afba0eaf.svg
cientgu
user
cientgu
6581348660ccc7f2302a767b
Chunyu Wang
6581348660ccc7f2302a767c
Ting Zhang
admin_assigned
2023-12-19T09:56:04.548Z
6350ffa18445bbe32e926360
/avatars/bc67d114191be7b16b463731a84c7e05.svg
Ting Zhang
user
hellozt
6581348660ccc7f2302a767d
Jianmin Bao
admin_assigned
2023-12-19T09:56:29.878Z
6481e690f9ed842838a2b106
/avatars/e89a3c8366df504a95dc08a1a412bf3d.svg
Jianmin Bao
user
jianmin-ustc
6581348660ccc7f2302a767e
Dong Chen
6581348660ccc7f2302a767f
Baining Guo
VolumeDiffusion: Flexible Text-to-3D Generation with Efficient
  Volumetric Encoder
This paper introduces a pioneering 3D volumetric encoder designed for
text-to-3D generation. To scale up the training data for the diffusion model, a
lightweight network is developed to efficiently acquire feature volumes from
multi-view images. The 3D volumes are then trained on a diffusion model for
text-to-3D generation using a 3D U-Net. This research further addresses the
challenges of inaccurate object captions and high-dimensional feature volumes.
The proposed model, trained on the public Objaverse dataset, demonstrates
promising outcomes in producing diverse and recognizable samples from text
prompts. Notably, it empowers finer control over object part characteristics
through textual cues, fostering model creativity by seamlessly combining
multiple concepts within a single object. This research significantly
contributes to the progress of 3D generation by introducing an efficient,
flexible, and scalable representation methodology. Code is available at
https://github.com/tzco/VolumeDiffusion.
A 3D volumetric encoder using a U-Net for diffusion models efficiently generates 3D objects from text prompts, addressing caption inaccuracies and providing finer control over object features.
3D volumetric encoder diffusion model lightweight network feature volumes multi-view images 3D U-Net text-to-3D generation Objaverse dataset diverse samples object part characteristics textual cues model creativity