DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction
  Model
We propose DMV3D, a novel 3D generation approach that uses a
transformer-based 3D large reconstruction model to denoise multi-view
diffusion. Our reconstruction model incorporates a triplane NeRF representation
and can denoise noisy multi-view images via NeRF reconstruction and rendering,
achieving single-stage 3D generation in sim30s on single A100 GPU. We train
DMV3D on large-scale multi-view image datasets of highly diverse
objects using only image reconstruction losses, without accessing 3D assets. We
demonstrate state-of-the-art results for the single-image reconstruction
problem where probabilistic modeling of unseen object parts is required for
generating diverse reconstructions with sharp textures. We also show
high-quality text-to-3D generation results outperforming previous 3D diffusion
models. Our project website is at: https://justimyhxu.github.io/projects/dmv3d/ .
6555ad0bf96492362e33e564
Yinghao Xu
claimed_verified
2024-03-01T09:28:21.740Z
6555da9adcf410fd0753569c
/avatars/ac58a796bb54f334fdb475a4b75c4d27.svg
Yinghao Xu
user
justimyhxu
6555ad0bf96492362e33e565
Hao Tan
admin_assigned
2023-11-16T08:30:14.956Z
5f297389e923d665e61627f9
/avatars/c39652c5b9ee182f4ea4dcd2f528f61b.svg
Hao Tan
user
airsplay
6555ad0bf96492362e33e566
Fujun Luan
6555ad0bf96492362e33e567
Sai Bi
6555ad0bf96492362e33e568
Peng Wang
admin_assigned
2023-11-16T08:31:33.365Z
6486ad27d29294981d2a7ebd
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/w-PUHQzMikeCU4matUio_.png
Peng Wang
user
totoro97
6555ad0bf96492362e33e569
Jiahao Li
extracted_pending
2023-11-16T05:47:59.116Z
630bffcf8b327c7b8b9dfcce
/avatars/12691f869f01217f7b92a326c30716b1.svg
Jiahao Li
user
54jiahao
6555ad0bf96492362e33e56a
Zifan Shi
6555ad0bf96492362e33e56b
Kalyan Sunkavalli
6555ad0bf96492362e33e56c
Gordon Wetzstein
6555ad0bf96492362e33e56d
Zexiang Xu
claimed_verified
2023-11-16T20:01:33.141Z
65565b1b8616d8d740abb976
/avatars/c245b19ed05292bf1bf4a9d8cf2a409c.svg
Zexiang Xu
user
zexiangxu
6555ad0bf96492362e33e56e
Kai Zhang
admin_assigned
2023-11-16T08:33:10.095Z
6351a882a4845f26d16a28d7
/avatars/4f8629fbf22faeb6511f4068f2828809.svg
Kai Zhang
user
Kai-46
DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction
  Model
We propose DMV3D, a novel 3D generation approach that uses a
transformer-based 3D large reconstruction model to denoise multi-view
diffusion. Our reconstruction model incorporates a triplane NeRF representation
and can denoise noisy multi-view images via NeRF reconstruction and rendering,
achieving single-stage 3D generation in sim30s on single A100 GPU. We train
DMV3D on large-scale multi-view image datasets of highly diverse
objects using only image reconstruction losses, without accessing 3D assets. We
demonstrate state-of-the-art results for the single-image reconstruction
problem where probabilistic modeling of unseen object parts is required for
generating diverse reconstructions with sharp textures. We also show
high-quality text-to-3D generation results outperforming previous 3D diffusion
models. Our project website is at: https://justimyhxu.github.io/projects/dmv3d/ .
DMV3D, a transformer-based 3D generation model using a triplane NeRF representation, achieves state-of-the-art results in single-image and text-to-3D reconstruction tasks within 30 seconds.
DMV3D transformer-based 3D generation denoise multi-view diffusion triplane NeRF NeRF image reconstruction losses single-image reconstruction probabilistic modeling text-to-3D generation