title: ReconFusion: 3D Reconstruction with Diffusion Priors
summary: 3D reconstruction methods such as Neural Radiance Fields (NeRFs) excel at
  rendering photorealistic novel views of complex scenes. However, recovering a
  high-quality NeRF typically requires tens to hundreds of input images,
  resulting in a time-consuming capture process. We present ReconFusion to
  reconstruct real-world scenes using only a few photos. Our approach leverages a
  diffusion prior for novel view synthesis, trained on synthetic and multiview
  datasets, which regularizes a NeRF-based 3D reconstruction pipeline at novel
  camera poses beyond those captured by the set of input images. Our method
  synthesizes realistic geometry and texture in underconstrained regions while
  preserving the appearance of observed regions. We perform an extensive
  evaluation across various real-world datasets, including forward-facing and
  360-degree scenes, demonstrating significant performance improvements over
  previous few-view NeRF reconstruction approaches.
paper_authors._id: 656fe23ae542334e137088c0
paper_authors.name: Rundi Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-06T12:47:00.925Z
paper_authors._id: 63823d59a18d1ca09653aa21
paper_authors.avatarUrl: /avatars/151e1ca46627abdd99c27945ea2fa441.svg
paper_authors.fullname: Rundi Wu
paper_authors.type: user
paper_authors.user: rundi
paper_authors._id: 656fe23ae542334e137088c1
paper_authors.name: Ben Mildenhall
paper_authors._id: 656fe23ae542334e137088c2
paper_authors.name: Philipp Henzler
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-06T12:47:20.237Z
paper_authors._id: 6386cc1fb3ef0523c6410261
paper_authors.avatarUrl: /avatars/f3e950e9044345130becd4b888d488a0.svg
paper_authors.fullname: P Henzler
paper_authors.type: user
paper_authors.user: solidst8ip
paper_authors._id: 656fe23ae542334e137088c3
paper_authors.name: Keunhong Park
paper_authors._id: 656fe23ae542334e137088c4
paper_authors.name: Ruiqi Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-06T12:47:40.628Z
paper_authors._id: 6137d46d1c0a1ca53676925f
paper_authors.avatarUrl: /avatars/f3d884a3971a858adc49e0f3b1932078.svg
paper_authors.fullname: Gao
paper_authors.type: user
paper_authors.user: Ruiqi
paper_authors._id: 656fe23ae542334e137088c5
paper_authors.name: Daniel Watson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-06T12:47:47.781Z
paper_authors._id: 644f2bb667c9458c913248f8
paper_authors.avatarUrl: /avatars/4065b41f7f3568def4a63c44c10670c4.svg
paper_authors.fullname: Daniel Watson
paper_authors.type: user
paper_authors.user: neftor
paper_authors._id: 656fe23ae542334e137088c6
paper_authors.name: Pratul P. Srinivasan
paper_authors._id: 656fe23ae542334e137088c7
paper_authors.name: Dor Verbin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-06T12:48:01.130Z
paper_authors._id: 6315b0fc915d0b806830b424
paper_authors.avatarUrl: /avatars/e25a7c8df2364c6e8024ec5c646a2889.svg
paper_authors.fullname: Dor Verbin
paper_authors.type: user
paper_authors.user: dorverbin
paper_authors._id: 656fe23ae542334e137088c8
paper_authors.name: Jonathan T. Barron
paper_authors._id: 656fe23ae542334e137088c9
paper_authors.name: Ben Poole
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-06T12:48:20.557Z
paper_authors._id: 63213768d2d45f315183abc3
paper_authors.avatarUrl: /avatars/521f52f430b73e514afb06e3a9d6779d.svg
paper_authors.fullname: Ben Poole
paper_authors.type: user
paper_authors.user: doinkda
paper_authors._id: 656fe23ae542334e137088ca
paper_authors.name: Aleksander Holynski
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-06T12:48:26.923Z
paper_authors._id: 62f6e2792e53c2efd33faa92
paper_authors.avatarUrl: /avatars/85d94b4022577747b8d2d10a82c2f3c7.svg
paper_authors.fullname: Aleksander Holynski
paper_authors.type: user
paper_authors.user: holynski
paper_title: ReconFusion: 3D Reconstruction with Diffusion Priors
paper_summary: 3D reconstruction methods such as Neural Radiance Fields (NeRFs) excel at
  rendering photorealistic novel views of complex scenes. However, recovering a
  high-quality NeRF typically requires tens to hundreds of input images,
  resulting in a time-consuming capture process. We present ReconFusion to
  reconstruct real-world scenes using only a few photos. Our approach leverages a
  diffusion prior for novel view synthesis, trained on synthetic and multiview
  datasets, which regularizes a NeRF-based 3D reconstruction pipeline at novel
  camera poses beyond those captured by the set of input images. Our method
  synthesizes realistic geometry and texture in underconstrained regions while
  preserving the appearance of observed regions. We perform an extensive
  evaluation across various real-world datasets, including forward-facing and
  360-degree scenes, demonstrating significant performance improvements over
  previous few-view NeRF reconstruction approaches.
paper_ai_summary: ReconFusion uses a diffusion prior to synthesize realistic geometry and texture with sparse input images for 3D reconstruction, outperforming previous few-view NeRF methods.
paper_ai_keywords: Neural Radiance Fields diffusion prior novel view synthesis 3D reconstruction underconstrained regions observed regions forward-facing 360-degree scenes