MicroDreamer: Zero-shot 3D Generation in sim20 Seconds by Score-based
  Iterative Reconstruction
Optimization-based approaches, such as score distillation sampling (SDS),
show promise in zero-shot 3D generation but suffer from low efficiency,
primarily due to the high number of function evaluations (NFEs) required for
each sample. In this paper, we introduce score-based iterative reconstruction
(SIR), an efficient and general algorithm for 3D generation with a multi-view
score-based diffusion model. Given the images produced by the diffusion model,
SIR reduces NFEs by repeatedly optimizing 3D parameters, unlike the single
optimization in SDS, mimicking the 3D reconstruction process. With other
improvements including optimization in the pixel space, we present an efficient
approach called MicroDreamer that generally applies to various 3D
representations and 3D generation tasks. In particular, retaining a comparable
performance, MicroDreamer is 5-20 times faster than SDS in generating neural
radiance field and takes about 20 seconds to generate meshes from 3D Gaussian
splitting on a single A100 GPU, halving the time of the fastest zero-shot
baseline, DreamGaussian. Our code is available at
https://github.com/ML-GSAI/MicroDreamer.
6631c599cc6a204ac6351734
Luxi Chen
6631c599cc6a204ac6351735
Zhengyi Wang
admin_assigned
2024-05-02T08:29:44.275Z
634e15aec1ce28f1de91c470
https://cdn-avatars.huggingface.co/v1/production/uploads/634e15aec1ce28f1de91c470/MdtJKgE_fbHydhqESr-DH.png
Zhengyi Wang
user
Zhengyi
6631c599cc6a204ac6351736
Chongxuan Li
admin_assigned
2024-05-02T08:29:49.940Z
64c07b488e2612254361153b
/avatars/ade0f783cc4c2d3e73f402637f595471.svg
chongxuan li
user
zhenxuan00
6631c599cc6a204ac6351737
Tingting Gao
admin_assigned
2024-05-02T08:29:56.380Z
656453832bdaccfcd5379431
/avatars/a0d764ce6b3fd05532c7a9cb2f263e33.svg
Gao Ting
user
TingTingGao
6631c599cc6a204ac6351738
Hang Su
6631c599cc6a204ac6351739
Jun Zhu
MicroDreamer: Zero-shot 3D Generation in sim20 Seconds by Score-based
  Iterative Reconstruction
Optimization-based approaches, such as score distillation sampling (SDS),
show promise in zero-shot 3D generation but suffer from low efficiency,
primarily due to the high number of function evaluations (NFEs) required for
each sample. In this paper, we introduce score-based iterative reconstruction
(SIR), an efficient and general algorithm for 3D generation with a multi-view
score-based diffusion model. Given the images produced by the diffusion model,
SIR reduces NFEs by repeatedly optimizing 3D parameters, unlike the single
optimization in SDS, mimicking the 3D reconstruction process. With other
improvements including optimization in the pixel space, we present an efficient
approach called MicroDreamer that generally applies to various 3D
representations and 3D generation tasks. In particular, retaining a comparable
performance, MicroDreamer is 5-20 times faster than SDS in generating neural
radiance field and takes about 20 seconds to generate meshes from 3D Gaussian
splitting on a single A100 GPU, halving the time of the fastest zero-shot
baseline, DreamGaussian. Our code is available at
https://github.com/ML-GSAI/MicroDreamer.
Score-based iterative reconstruction (SIR) enhances zero-shot 3D generation efficiency compared to score distillation sampling by optimizing 3D parameters iteratively, reducing the number of function evaluations and achieving faster generation times.
score distillation sampling score-based iterative reconstruction multi-view score-based diffusion model neural radiance field 3D Gaussian splitting MicroDreamer DreamGaussian