title: MVDD: Multi-View Depth Diffusion Models
summary: Denoising diffusion models have demonstrated outstanding results in 2D image
  generation, yet it remains a challenge to replicate its success in 3D shape
  generation. In this paper, we propose leveraging multi-view depth, which
  represents complex 3D shapes in a 2D data format that is easy to denoise. We
  pair this representation with a diffusion model, MVDD, that is capable of
  generating high-quality dense point clouds with 20K+ points with fine-grained
  details. To enforce 3D consistency in multi-view depth, we introduce an
  epipolar line segment attention that conditions the denoising step for a view
  on its neighboring views. Additionally, a depth fusion module is incorporated
  into diffusion steps to further ensure the alignment of depth maps. When
  augmented with surface reconstruction, MVDD can also produce high-quality 3D
  meshes. Furthermore, MVDD stands out in other tasks such as depth completion,
  and can serve as a 3D prior, significantly boosting many downstream tasks, such
  as GAN inversion. State-of-the-art results from extensive experiments
  demonstrate MVDD's excellent ability in 3D shape generation, depth completion,
  and its potential as a 3D prior for downstream tasks.
paper_authors._id: 65768052412ee701850faa52
paper_authors.name: Zhen Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-11T08:51:14.418Z
paper_authors._id: 603e75e27bfa033a4f7ca56f
paper_authors.avatarUrl: /avatars/d1be56fa7e6ffe77f4acd2f6e3c6a250.svg
paper_authors.fullname: wang
paper_authors.type: user
paper_authors.user: zhen
paper_authors._id: 65768052412ee701850faa53
paper_authors.name: Qiangeng Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-11T08:51:21.344Z
paper_authors._id: 64c1a214c84cd12dd79c1530
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/CSK0AyQhXs3Izrnt9s6_l.png
paper_authors.fullname: qiangeng xu
paper_authors.type: user
paper_authors.user: xharlie
paper_authors._id: 65768052412ee701850faa54
paper_authors.name: Feitong Tan
paper_authors._id: 65768052412ee701850faa55
paper_authors.name: Menglei Chai
paper_authors._id: 65768052412ee701850faa56
paper_authors.name: Shichen Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-11T08:51:53.406Z
paper_authors._id: 5f8d08035d083370c711f4f1
paper_authors.avatarUrl: /avatars/3d5b151c937774eb51af25814202c4c4.svg
paper_authors.fullname: LiuShiChen
paper_authors.type: user
paper_authors.user: schen
paper_authors._id: 65768052412ee701850faa57
paper_authors.name: Rohit Pandey
paper_authors._id: 65768052412ee701850faa58
paper_authors.name: Sean Fanello
paper_authors._id: 65768052412ee701850faa59
paper_authors.name: Achuta Kadambi
paper_authors._id: 65768052412ee701850faa5a
paper_authors.name: Yinda Zhang
paper_title: MVDD: Multi-View Depth Diffusion Models
paper_summary: Denoising diffusion models have demonstrated outstanding results in 2D image
  generation, yet it remains a challenge to replicate its success in 3D shape
  generation. In this paper, we propose leveraging multi-view depth, which
  represents complex 3D shapes in a 2D data format that is easy to denoise. We
  pair this representation with a diffusion model, MVDD, that is capable of
  generating high-quality dense point clouds with 20K+ points with fine-grained
  details. To enforce 3D consistency in multi-view depth, we introduce an
  epipolar line segment attention that conditions the denoising step for a view
  on its neighboring views. Additionally, a depth fusion module is incorporated
  into diffusion steps to further ensure the alignment of depth maps. When
  augmented with surface reconstruction, MVDD can also produce high-quality 3D
  meshes. Furthermore, MVDD stands out in other tasks such as depth completion,
  and can serve as a 3D prior, significantly boosting many downstream tasks, such
  as GAN inversion. State-of-the-art results from extensive experiments
  demonstrate MVDD's excellent ability in 3D shape generation, depth completion,
  and its potential as a 3D prior for downstream tasks.
paper_ai_summary: MVDD is a denoising diffusion model using multi-view depth representations to generate high-quality 3D shape point clouds and meshes, achieving state-of-the-art results in 3D shape generation and depth completion.
paper_ai_keywords: denoising diffusion models multi-view depth epipolar line segment attention depth fusion module surface reconstruction depth completion GAN inversion 3D shape generation 3D prior