title: MVD^2: Efficient Multiview 3D Reconstruction for Multiview Diffusion
summary: As a promising 3D generation technique, multiview diffusion (MVD) has
  received a lot of attention due to its advantages in terms of generalizability,
  quality, and efficiency. By finetuning pretrained large image diffusion models
  with 3D data, the MVD methods first generate multiple views of a 3D object
  based on an image or text prompt and then reconstruct 3D shapes with multiview
  3D reconstruction. However, the sparse views and inconsistent details in the
  generated images make 3D reconstruction challenging. We present MVD^2, an
  efficient 3D reconstruction method for multiview diffusion (MVD) images.
  MVD^2 aggregates image features into a 3D feature volume by projection and
  convolution and then decodes volumetric features into a 3D mesh. We train
  MVD^2 with 3D shape collections and MVD images prompted by rendered views of
  3D shapes. To address the discrepancy between the generated multiview images
  and ground-truth views of the 3D shapes, we design a simple-yet-efficient
  view-dependent training scheme. MVD^2 improves the 3D generation quality of
  MVD and is fast and robust to various MVD methods. After training, it can
  efficiently decode 3D meshes from multiview images within one second. We train
  MVD^2 with Zero-123++ and ObjectVerse-LVIS 3D dataset and demonstrate its
  superior performance in generating 3D models from multiview images generated by
  different MVD methods, using both synthetic and real images as prompts.
paper_authors._id: 65d809be685624d5f2e51e05
paper_authors.name: Xin-Yang Zheng
paper_authors._id: 65d809be685624d5f2e51e06
paper_authors.name: Hao Pan
paper_authors._id: 65d809be685624d5f2e51e07
paper_authors.name: Yu-Xiao Guo
paper_authors._id: 65d809be685624d5f2e51e08
paper_authors.name: Xin Tong
paper_authors._id: 65d809be685624d5f2e51e09
paper_authors.name: Yang Liu
paper_title: MVD^2: Efficient Multiview 3D Reconstruction for Multiview Diffusion
paper_summary: As a promising 3D generation technique, multiview diffusion (MVD) has
  received a lot of attention due to its advantages in terms of generalizability,
  quality, and efficiency. By finetuning pretrained large image diffusion models
  with 3D data, the MVD methods first generate multiple views of a 3D object
  based on an image or text prompt and then reconstruct 3D shapes with multiview
  3D reconstruction. However, the sparse views and inconsistent details in the
  generated images make 3D reconstruction challenging. We present MVD^2, an
  efficient 3D reconstruction method for multiview diffusion (MVD) images.
  MVD^2 aggregates image features into a 3D feature volume by projection and
  convolution and then decodes volumetric features into a 3D mesh. We train
  MVD^2 with 3D shape collections and MVD images prompted by rendered views of
  3D shapes. To address the discrepancy between the generated multiview images
  and ground-truth views of the 3D shapes, we design a simple-yet-efficient
  view-dependent training scheme. MVD^2 improves the 3D generation quality of
  MVD and is fast and robust to various MVD methods. After training, it can
  efficiently decode 3D meshes from multiview images within one second. We train
  MVD^2 with Zero-123++ and ObjectVerse-LVIS 3D dataset and demonstrate its
  superior performance in generating 3D models from multiview images generated by
  different MVD methods, using both synthetic and real images as prompts.
paper_ai_summary: MVD$^2$ enhances 3D reconstruction by aggregating image features into a 3D volume and efficiently decoding them into meshes, improving the quality and robustness of multiview diffusion-based 3D generation.
paper_ai_keywords: multiview diffusion MVD 3D reconstruction 3D feature volume projection convolution 3D mesh view-dependent training Zero-123++ ObjectVerse-LVIS 3D dataset