title: UniSDF: Unifying Neural Representations for High-Fidelity 3D
    Reconstruction of Complex Scenes with Reflections
summary: Neural 3D scene representations have shown great potential for 3D
  reconstruction from 2D images. However, reconstructing real-world captures of
  complex scenes still remains a challenge. Existing generic 3D reconstruction
  methods often struggle to represent fine geometric details and do not
  adequately model reflective surfaces of large-scale scenes. Techniques that
  explicitly focus on reflective surfaces can model complex and detailed
  reflections by exploiting better reflection parameterizations. However, we
  observe that these methods are often not robust in real unbounded scenarios
  where non-reflective as well as reflective components are present. In this
  work, we propose UniSDF, a general purpose 3D reconstruction method that can
  reconstruct large complex scenes with reflections. We investigate both
  view-based as well as reflection-based color prediction parameterization
  techniques and find that explicitly blending these representations in 3D space
  enables reconstruction of surfaces that are more geometrically accurate,
  especially for reflective surfaces. We further combine this representation with
  a multi-resolution grid backbone that is trained in a coarse-to-fine manner,
  enabling faster reconstructions than prior methods. Extensive experiments on
  object-level datasets DTU, Shiny Blender as well as unbounded datasets Mip-NeRF
  360 and Ref-NeRF real demonstrate that our method is able to robustly
  reconstruct complex large-scale scenes with fine details and reflective
  surfaces. Please see our project page at
  https://fangjinhuawang.github.io/UniSDF.
paper_authors._id: 6583bce1af09ca206bcda61b
paper_authors.name: Fangjinhua Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-21T09:56:37.576Z
paper_authors._id: 6584099fcce29a06215a5839
paper_authors.avatarUrl: /avatars/8bc526b953f8ddf91a79cf210ee1c4bd.svg
paper_authors.fullname: Fangjinhua Wang
paper_authors.type: user
paper_authors.user: fangjinhuawang
paper_authors._id: 6583bce1af09ca206bcda61c
paper_authors.name: Marie-Julie Rakotosaona
paper_authors._id: 6583bce1af09ca206bcda61d
paper_authors.name: Michael Niemeyer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T09:40:41.228Z
paper_authors._id: 63f4f16171a5d395c71d2d1a
paper_authors.avatarUrl: /avatars/0377c4e6b35f8bd1a29d38c99a5943c1.svg
paper_authors.fullname: Michael Niemeyer
paper_authors.type: user
paper_authors.user: mniemeyer
paper_authors._id: 6583bce1af09ca206bcda61e
paper_authors.name: Richard Szeliski
paper_authors._id: 6583bce1af09ca206bcda61f
paper_authors.name: Marc Pollefeys
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-25T09:43:30.609Z
paper_authors._id: 67b5fa179782a5e2fd2cb26a
paper_authors.avatarUrl: /avatars/62c38f29ec641e001eeddf840bea21a0.svg
paper_authors.fullname: Marc Pollefeys
paper_authors.type: user
paper_authors.user: mapo1
paper_authors._id: 6583bce1af09ca206bcda620
paper_authors.name: Federico Tombari
paper_title: UniSDF: Unifying Neural Representations for High-Fidelity 3D
    Reconstruction of Complex Scenes with Reflections
paper_summary: Neural 3D scene representations have shown great potential for 3D
  reconstruction from 2D images. However, reconstructing real-world captures of
  complex scenes still remains a challenge. Existing generic 3D reconstruction
  methods often struggle to represent fine geometric details and do not
  adequately model reflective surfaces of large-scale scenes. Techniques that
  explicitly focus on reflective surfaces can model complex and detailed
  reflections by exploiting better reflection parameterizations. However, we
  observe that these methods are often not robust in real unbounded scenarios
  where non-reflective as well as reflective components are present. In this
  work, we propose UniSDF, a general purpose 3D reconstruction method that can
  reconstruct large complex scenes with reflections. We investigate both
  view-based as well as reflection-based color prediction parameterization
  techniques and find that explicitly blending these representations in 3D space
  enables reconstruction of surfaces that are more geometrically accurate,
  especially for reflective surfaces. We further combine this representation with
  a multi-resolution grid backbone that is trained in a coarse-to-fine manner,
  enabling faster reconstructions than prior methods. Extensive experiments on
  object-level datasets DTU, Shiny Blender as well as unbounded datasets Mip-NeRF
  360 and Ref-NeRF real demonstrate that our method is able to robustly
  reconstruct complex large-scale scenes with fine details and reflective
  surfaces. Please see our project page at
  https://fangjinhuawang.github.io/UniSDF.
paper_ai_keywords: Neural 3D scene representations 3D reconstruction 2D images fine geometric details reflective surfaces reflection parameterizations view-based color prediction reflection-based color prediction explicit blending 3D space multi-resolution grid backbone coarse-to-fine manner DTU Shiny Blender Mip-NeRF 360 Ref-NeRF real reconstruct complex large-scale scenes