title: MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for
    Single or Sparse-view 3D Object Reconstruction
summary: This paper presents a neural architecture MVDiffusion++ for 3D object
  reconstruction that synthesizes dense and high-resolution views of an object
  given one or a few images without camera poses. MVDiffusion++ achieves superior
  flexibility and scalability with two surprisingly simple ideas: 1) A
  ``pose-free architecture'' where standard self-attention among 2D latent
  features learns 3D consistency across an arbitrary number of conditional and
  generation views without explicitly using camera pose information; and 2) A
  ``view dropout strategy'' that discards a substantial number of output views
  during training, which reduces the training-time memory footprint and enables
  dense and high-resolution view synthesis at test time. We use the Objaverse for
  training and the Google Scanned Objects for evaluation with standard novel view
  synthesis and 3D reconstruction metrics, where MVDiffusion++ significantly
  outperforms the current state of the arts. We also demonstrate a text-to-3D
  application example by combining MVDiffusion++ with a text-to-image generative
  model.
paper_authors._id: 65d56914a71cb46f50cfc21a
paper_authors.name: Shitao Tang
paper_authors._id: 65d56914a71cb46f50cfc21b
paper_authors.name: Jiacheng Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-04T07:32:36.151Z
paper_authors._id: 655bca95360e4f90cb61ba83
paper_authors.avatarUrl: /avatars/1a187beb91a5e2fdc2303620b742aab1.svg
paper_authors.fullname: Jiacheng Chen
paper_authors.type: user
paper_authors.user: cccjc
paper_authors._id: 65d56914a71cb46f50cfc21c
paper_authors.name: Dilin Wang
paper_authors._id: 65d56914a71cb46f50cfc21d
paper_authors.name: Chengzhou Tang
paper_authors._id: 65d56914a71cb46f50cfc21e
paper_authors.name: Fuyang Zhang
paper_authors._id: 65d56914a71cb46f50cfc21f
paper_authors.name: Yuchen Fan
paper_authors._id: 65d56914a71cb46f50cfc220
paper_authors.name: Vikas Chandra
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-21T10:31:48.934Z
paper_authors._id: 6566681f7b5ed0735812af32
paper_authors.avatarUrl: /avatars/424905abb2973954b0850c592743b6fb.svg
paper_authors.fullname: Vikas Chandra
paper_authors.type: user
paper_authors.user: vchandra
paper_authors._id: 65d56914a71cb46f50cfc221
paper_authors.name: Yasutaka Furukawa
paper_authors._id: 65d56914a71cb46f50cfc222
paper_authors.name: Rakesh Ranjan
paper_title: MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for
    Single or Sparse-view 3D Object Reconstruction
paper_summary: This paper presents a neural architecture MVDiffusion++ for 3D object
  reconstruction that synthesizes dense and high-resolution views of an object
  given one or a few images without camera poses. MVDiffusion++ achieves superior
  flexibility and scalability with two surprisingly simple ideas: 1) A
  ``pose-free architecture'' where standard self-attention among 2D latent
  features learns 3D consistency across an arbitrary number of conditional and
  generation views without explicitly using camera pose information; and 2) A
  ``view dropout strategy'' that discards a substantial number of output views
  during training, which reduces the training-time memory footprint and enables
  dense and high-resolution view synthesis at test time. We use the Objaverse for
  training and the Google Scanned Objects for evaluation with standard novel view
  synthesis and 3D reconstruction metrics, where MVDiffusion++ significantly
  outperforms the current state of the arts. We also demonstrate a text-to-3D
  application example by combining MVDiffusion++ with a text-to-image generative
  model.
paper_ai_keywords: MVDiffusion++ 3D object reconstruction dense and high-resolution views self-attention 2D latent features 3D consistency pose-free architecture conditional views generation views view dropout strategy training-time memory footprint novel view synthesis 3D reconstruction metrics text-to-3D application text-to-image generative model