MVDD: Multi-View Depth Diffusion Models
Denoising diffusion models have demonstrated outstanding results in 2D image
generation, yet it remains a challenge to replicate its success in 3D shape
generation. In this paper, we propose leveraging multi-view depth, which
represents complex 3D shapes in a 2D data format that is easy to denoise. We
pair this representation with a diffusion model, MVDD, that is capable of
generating high-quality dense point clouds with 20K+ points with fine-grained
details. To enforce 3D consistency in multi-view depth, we introduce an
epipolar line segment attention that conditions the denoising step for a view
on its neighboring views. Additionally, a depth fusion module is incorporated
into diffusion steps to further ensure the alignment of depth maps. When
augmented with surface reconstruction, MVDD can also produce high-quality 3D
meshes. Furthermore, MVDD stands out in other tasks such as depth completion,
and can serve as a 3D prior, significantly boosting many downstream tasks, such
as GAN inversion. State-of-the-art results from extensive experiments
demonstrate MVDD's excellent ability in 3D shape generation, depth completion,
and its potential as a 3D prior for downstream tasks.
65768052412ee701850faa52
Zhen Wang
admin_assigned
2023-12-11T08:51:14.418Z
603e75e27bfa033a4f7ca56f
/avatars/d1be56fa7e6ffe77f4acd2f6e3c6a250.svg
wang
user
zhen
65768052412ee701850faa53
Qiangeng Xu
admin_assigned
2023-12-11T08:51:21.344Z
64c1a214c84cd12dd79c1530
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/CSK0AyQhXs3Izrnt9s6_l.png
qiangeng xu
user
xharlie
65768052412ee701850faa54
Feitong Tan
65768052412ee701850faa55
Menglei Chai
65768052412ee701850faa56
Shichen Liu
admin_assigned
2023-12-11T08:51:53.406Z
5f8d08035d083370c711f4f1
/avatars/3d5b151c937774eb51af25814202c4c4.svg
LiuShiChen
user
schen
65768052412ee701850faa57
Rohit Pandey
65768052412ee701850faa58
Sean Fanello
65768052412ee701850faa59
Achuta Kadambi
65768052412ee701850faa5a
Yinda Zhang
MVDD: Multi-View Depth Diffusion Models
Denoising diffusion models have demonstrated outstanding results in 2D image
generation, yet it remains a challenge to replicate its success in 3D shape
generation. In this paper, we propose leveraging multi-view depth, which
represents complex 3D shapes in a 2D data format that is easy to denoise. We
pair this representation with a diffusion model, MVDD, that is capable of
generating high-quality dense point clouds with 20K+ points with fine-grained
details. To enforce 3D consistency in multi-view depth, we introduce an
epipolar line segment attention that conditions the denoising step for a view
on its neighboring views. Additionally, a depth fusion module is incorporated
into diffusion steps to further ensure the alignment of depth maps. When
augmented with surface reconstruction, MVDD can also produce high-quality 3D
meshes. Furthermore, MVDD stands out in other tasks such as depth completion,
and can serve as a 3D prior, significantly boosting many downstream tasks, such
as GAN inversion. State-of-the-art results from extensive experiments
demonstrate MVDD's excellent ability in 3D shape generation, depth completion,
and its potential as a 3D prior for downstream tasks.
MVDD is a denoising diffusion model using multi-view depth representations to generate high-quality 3D shape point clouds and meshes, achieving state-of-the-art results in 3D shape generation and depth completion.
denoising diffusion models multi-view depth epipolar line segment attention depth fusion module surface reconstruction depth completion GAN inversion 3D shape generation 3D prior