title: Adaptive Shells for Efficient Neural Radiance Field Rendering
summary: Neural radiance fields achieve unprecedented quality for novel view
  synthesis, but their volumetric formulation remains expensive, requiring a huge
  number of samples to render high-resolution images. Volumetric encodings are
  essential to represent fuzzy geometry such as foliage and hair, and they are
  well-suited for stochastic optimization. Yet, many scenes ultimately consist
  largely of solid surfaces which can be accurately rendered by a single sample
  per pixel. Based on this insight, we propose a neural radiance formulation that
  smoothly transitions between volumetric- and surface-based rendering, greatly
  accelerating rendering speed and even improving visual fidelity. Our method
  constructs an explicit mesh envelope which spatially bounds a neural volumetric
  representation. In solid regions, the envelope nearly converges to a surface
  and can often be rendered with a single sample. To this end, we generalize the
  NeuS formulation with a learned spatially-varying kernel size which encodes the
  spread of the density, fitting a wide kernel to volume-like regions and a tight
  kernel to surface-like regions. We then extract an explicit mesh of a narrow
  band around the surface, with width determined by the kernel size, and
  fine-tune the radiance field within this band. At inference time, we cast rays
  against the mesh and evaluate the radiance field only within the enclosed
  region, greatly reducing the number of samples required. Experiments show that
  our approach enables efficient rendering at very high fidelity. We also
  demonstrate that the extracted envelope enables downstream applications such as
  animation and simulation.
paper_authors._id: 6556e9a0df3c800e012583ba
paper_authors.name: Zian Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-21T10:24:43.245Z
paper_authors._id: 655be57371ed242b42600a58
paper_authors.avatarUrl: /avatars/a7199cf02bb136fa95c7de89b627871f.svg
paper_authors.fullname: Zian Wang
paper_authors.type: user
paper_authors.user: zianw
paper_authors._id: 6556e9a0df3c800e012583bb
paper_authors.name: Tianchang Shen
paper_authors._id: 6556e9a0df3c800e012583bc
paper_authors.name: Merlin Nimier-David
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-17T16:14:49.855Z
paper_authors._id: 65579145976a138166b2f4e3
paper_authors.avatarUrl: /avatars/e042c25d55f9ef560fc0c2de51188b2d.svg
paper_authors.fullname: Merlin Nimier-David
paper_authors.type: user
paper_authors.user: merlinND
paper_authors._id: 6556e9a0df3c800e012583bd
paper_authors.name: Nicholas Sharp
paper_authors._id: 6556e9a0df3c800e012583be
paper_authors.name: Jun Gao
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-09-27T15:02:30.090Z
paper_authors._id: 66f6c8f8d9f80465ae1d671e
paper_authors.avatarUrl: /avatars/851500d2fee048c8b2b81d11b5429e11.svg
paper_authors.fullname: Gao
paper_authors.type: user
paper_authors.user: JunGaoNVIDIA
paper_authors._id: 6556e9a0df3c800e012583bf
paper_authors.name: Alexander Keller
paper_authors._id: 6556e9a0df3c800e012583c0
paper_authors.name: Sanja Fidler
paper_authors._id: 6556e9a0df3c800e012583c1
paper_authors.name: Thomas Müller
paper_authors._id: 6556e9a0df3c800e012583c2
paper_authors.name: Zan Gojcic
paper_title: Adaptive Shells for Efficient Neural Radiance Field Rendering
paper_summary: Neural radiance fields achieve unprecedented quality for novel view
  synthesis, but their volumetric formulation remains expensive, requiring a huge
  number of samples to render high-resolution images. Volumetric encodings are
  essential to represent fuzzy geometry such as foliage and hair, and they are
  well-suited for stochastic optimization. Yet, many scenes ultimately consist
  largely of solid surfaces which can be accurately rendered by a single sample
  per pixel. Based on this insight, we propose a neural radiance formulation that
  smoothly transitions between volumetric- and surface-based rendering, greatly
  accelerating rendering speed and even improving visual fidelity. Our method
  constructs an explicit mesh envelope which spatially bounds a neural volumetric
  representation. In solid regions, the envelope nearly converges to a surface
  and can often be rendered with a single sample. To this end, we generalize the
  NeuS formulation with a learned spatially-varying kernel size which encodes the
  spread of the density, fitting a wide kernel to volume-like regions and a tight
  kernel to surface-like regions. We then extract an explicit mesh of a narrow
  band around the surface, with width determined by the kernel size, and
  fine-tune the radiance field within this band. At inference time, we cast rays
  against the mesh and evaluate the radiance field only within the enclosed
  region, greatly reducing the number of samples required. Experiments show that
  our approach enables efficient rendering at very high fidelity. We also
  demonstrate that the extracted envelope enables downstream applications such as
  animation and simulation.
paper_ai_keywords: neural radiance fields novel view synthesis volumetric formulation rendering speed volumetric encodings stochastic optimization explicit mesh envelope neural volumetric representation surface-based rendering learned spatially-varying kernel size density spread kernel size explicit mesh narrow band high fidelity downstream applications animation simulation