title: PlatoNeRF: 3D Reconstruction in Plato's Cave via Single-View Two-Bounce
    Lidar
summary: 3D reconstruction from a single-view is challenging because of the ambiguity
  from monocular cues and lack of information about occluded regions. Neural
  radiance fields (NeRF), while popular for view synthesis and 3D reconstruction,
  are typically reliant on multi-view images. Existing methods for single-view 3D
  reconstruction with NeRF rely on either data priors to hallucinate views of
  occluded regions, which may not be physically accurate, or shadows observed by
  RGB cameras, which are difficult to detect in ambient light and low albedo
  backgrounds. We propose using time-of-flight data captured by a single-photon
  avalanche diode to overcome these limitations. Our method models two-bounce
  optical paths with NeRF, using lidar transient data for supervision. By
  leveraging the advantages of both NeRF and two-bounce light measured by lidar,
  we demonstrate that we can reconstruct visible and occluded geometry without
  data priors or reliance on controlled ambient lighting or scene albedo. In
  addition, we demonstrate improved generalization under practical constraints on
  sensor spatial- and temporal-resolution. We believe our method is a promising
  direction as single-photon lidars become ubiquitous on consumer devices, such
  as phones, tablets, and headsets.
paper_authors._id: 6588f61c067630f69f681217
paper_authors.name: Tzofi Klinghoffer
paper_authors._id: 6588f61c067630f69f681218
paper_authors.name: Xiaoyu Xiang
paper_authors._id: 6588f61c067630f69f681219
paper_authors.name: Siddharth Somasundaram
paper_authors._id: 6588f61c067630f69f68121a
paper_authors.name: Yuchen Fan
paper_authors._id: 6588f61c067630f69f68121b
paper_authors.name: Christian Richardt
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-13T20:36:56.127Z
paper_authors._id: 67adc0cfe1f8b9375a6c7a63
paper_authors.avatarUrl: /avatars/942d92ff9bd9e7a0f8f4687babbb8153.svg
paper_authors.fullname: Christian Richardt
paper_authors.type: user
paper_authors.user: cr333
paper_authors._id: 6588f61c067630f69f68121c
paper_authors.name: Ramesh Raskar
paper_authors._id: 6588f61c067630f69f68121d
paper_authors.name: Rakesh Ranjan
paper_title: PlatoNeRF: 3D Reconstruction in Plato's Cave via Single-View Two-Bounce
    Lidar
paper_summary: 3D reconstruction from a single-view is challenging because of the ambiguity
  from monocular cues and lack of information about occluded regions. Neural
  radiance fields (NeRF), while popular for view synthesis and 3D reconstruction,
  are typically reliant on multi-view images. Existing methods for single-view 3D
  reconstruction with NeRF rely on either data priors to hallucinate views of
  occluded regions, which may not be physically accurate, or shadows observed by
  RGB cameras, which are difficult to detect in ambient light and low albedo
  backgrounds. We propose using time-of-flight data captured by a single-photon
  avalanche diode to overcome these limitations. Our method models two-bounce
  optical paths with NeRF, using lidar transient data for supervision. By
  leveraging the advantages of both NeRF and two-bounce light measured by lidar,
  we demonstrate that we can reconstruct visible and occluded geometry without
  data priors or reliance on controlled ambient lighting or scene albedo. In
  addition, we demonstrate improved generalization under practical constraints on
  sensor spatial- and temporal-resolution. We believe our method is a promising
  direction as single-photon lidars become ubiquitous on consumer devices, such
  as phones, tablets, and headsets.
paper_ai_summary: A single-photon lidar-based approach combines NeRF and two-bounce light measurements to achieve accurate single-view 3D reconstruction without data priors or controlled lighting.
paper_ai_keywords: neural radiance fields NeRF time-of-flight data single-photon avalanche diode two-bounce optical paths lidar transient data occluded geometry sensor spatial-resolution sensor temporal-resolution