Make-A-Shape: a Ten-Million-scale 3D Shape Model
Significant progress has been made in training large generative models for
natural language and images. Yet, the advancement of 3D generative models is
hindered by their substantial resource demands for training, along with
inefficient, non-compact, and less expressive representations. This paper
introduces Make-A-Shape, a new 3D generative model designed for efficient
training on a vast scale, capable of utilizing 10 millions publicly-available
shapes. Technical-wise, we first innovate a wavelet-tree representation to
compactly encode shapes by formulating the subband coefficient filtering scheme
to efficiently exploit coefficient relations. We then make the representation
generatable by a diffusion model by devising the subband coefficients packing
scheme to layout the representation in a low-resolution grid. Further, we
derive the subband adaptive training strategy to train our model to effectively
learn to generate coarse and detail wavelet coefficients. Last, we extend our
framework to be controlled by additional input conditions to enable it to
generate shapes from assorted modalities, e.g., single/multi-view images, point
clouds, and low-resolution voxels. In our extensive set of experiments, we
demonstrate various applications, such as unconditional generation, shape
completion, and conditional generation on a wide range of modalities. Our
approach not only surpasses the state of the art in delivering high-quality
results but also efficiently generates shapes within a few seconds, often
achieving this in just 2 seconds for most conditions.
65af502248481b399a472331
Ka-Hei Hui
admin_assigned
2024-01-23T10:42:50.483Z
64d67c5aaa89fb548aa3a391
/avatars/971455c8465d4828aee2379f87c4b874.svg
Ka-Hei Hui
user
khhui
65af502248481b399a472332
Aditya Sanghi
65af502248481b399a472333
Arianna Rampini
admin_assigned
2024-01-23T10:43:18.119Z
653a83318f8d60f2046b27b1
/avatars/ab90522fa53738a6f67aca189ec45e6f.svg
Arianna Rampini
user
rampina
65af502248481b399a472334
Kamal Rahimi Malekshan
65af502248481b399a472335
Zhengzhe Liu
claimed_verified
2024-01-23T11:00:38.808Z
633e1588e7f8b20a2dce5b44
/avatars/8ce0db03b19c3970c0862c490485e831.svg
zzliu
user
xiaoxinge
65af502248481b399a472336
Hooman Shayani
65af502248481b399a472337
Chi-Wing Fu
Make-A-Shape: a Ten-Million-scale 3D Shape Model
Significant progress has been made in training large generative models for
natural language and images. Yet, the advancement of 3D generative models is
hindered by their substantial resource demands for training, along with
inefficient, non-compact, and less expressive representations. This paper
introduces Make-A-Shape, a new 3D generative model designed for efficient
training on a vast scale, capable of utilizing 10 millions publicly-available
shapes. Technical-wise, we first innovate a wavelet-tree representation to
compactly encode shapes by formulating the subband coefficient filtering scheme
to efficiently exploit coefficient relations. We then make the representation
generatable by a diffusion model by devising the subband coefficients packing
scheme to layout the representation in a low-resolution grid. Further, we
derive the subband adaptive training strategy to train our model to effectively
learn to generate coarse and detail wavelet coefficients. Last, we extend our
framework to be controlled by additional input conditions to enable it to
generate shapes from assorted modalities, e.g., single/multi-view images, point
clouds, and low-resolution voxels. In our extensive set of experiments, we
demonstrate various applications, such as unconditional generation, shape
completion, and conditional generation on a wide range of modalities. Our
approach not only surpasses the state of the art in delivering high-quality
results but also efficiently generates shapes within a few seconds, often
achieving this in just 2 seconds for most conditions.
Make-A-Shape, a novel 3D generative model, efficiently trains using wavelet-tree representation and diffusion model techniques, generating high-quality shapes from various inputs in under two seconds.
wavelet-tree representation diffusion model subband coefficient filtering subband coefficients packing subband adaptive training strategy unconditional generation shape completion conditional generation single/multi-view images point clouds low-resolution voxels