Adaptive Shells for Efficient Neural Radiance Field Rendering
Neural radiance fields achieve unprecedented quality for novel view
synthesis, but their volumetric formulation remains expensive, requiring a huge
number of samples to render high-resolution images. Volumetric encodings are
essential to represent fuzzy geometry such as foliage and hair, and they are
well-suited for stochastic optimization. Yet, many scenes ultimately consist
largely of solid surfaces which can be accurately rendered by a single sample
per pixel. Based on this insight, we propose a neural radiance formulation that
smoothly transitions between volumetric- and surface-based rendering, greatly
accelerating rendering speed and even improving visual fidelity. Our method
constructs an explicit mesh envelope which spatially bounds a neural volumetric
representation. In solid regions, the envelope nearly converges to a surface
and can often be rendered with a single sample. To this end, we generalize the
NeuS formulation with a learned spatially-varying kernel size which encodes the
spread of the density, fitting a wide kernel to volume-like regions and a tight
kernel to surface-like regions. We then extract an explicit mesh of a narrow
band around the surface, with width determined by the kernel size, and
fine-tune the radiance field within this band. At inference time, we cast rays
against the mesh and evaluate the radiance field only within the enclosed
region, greatly reducing the number of samples required. Experiments show that
our approach enables efficient rendering at very high fidelity. We also
demonstrate that the extracted envelope enables downstream applications such as
animation and simulation.
6556e9a0df3c800e012583ba
Zian Wang
claimed_verified
2023-11-21T10:24:43.245Z
655be57371ed242b42600a58
/avatars/a7199cf02bb136fa95c7de89b627871f.svg
Zian Wang
user
zianw
6556e9a0df3c800e012583bb
Tianchang Shen
6556e9a0df3c800e012583bc
Merlin Nimier-David
claimed_verified
2023-11-17T16:14:49.855Z
65579145976a138166b2f4e3
/avatars/e042c25d55f9ef560fc0c2de51188b2d.svg
Merlin Nimier-David
user
merlinND
6556e9a0df3c800e012583bd
Nicholas Sharp
6556e9a0df3c800e012583be
Jun Gao
extracted_pending
2024-09-27T15:02:30.090Z
66f6c8f8d9f80465ae1d671e
/avatars/851500d2fee048c8b2b81d11b5429e11.svg
Gao
user
JunGaoNVIDIA
6556e9a0df3c800e012583bf
Alexander Keller
6556e9a0df3c800e012583c0
Sanja Fidler
6556e9a0df3c800e012583c1
Thomas Müller
6556e9a0df3c800e012583c2
Zan Gojcic
claimed_verified
2025-03-26T20:51:19.145Z
6366cda3361a96184dc22139
/avatars/d8a88c84cb5f69e69dd038674a29be89.svg
Zan Gojcic
user
zgojcic
Adaptive Shells for Efficient Neural Radiance Field Rendering
Neural radiance fields achieve unprecedented quality for novel view
synthesis, but their volumetric formulation remains expensive, requiring a huge
number of samples to render high-resolution images. Volumetric encodings are
essential to represent fuzzy geometry such as foliage and hair, and they are
well-suited for stochastic optimization. Yet, many scenes ultimately consist
largely of solid surfaces which can be accurately rendered by a single sample
per pixel. Based on this insight, we propose a neural radiance formulation that
smoothly transitions between volumetric- and surface-based rendering, greatly
accelerating rendering speed and even improving visual fidelity. Our method
constructs an explicit mesh envelope which spatially bounds a neural volumetric
representation. In solid regions, the envelope nearly converges to a surface
and can often be rendered with a single sample. To this end, we generalize the
NeuS formulation with a learned spatially-varying kernel size which encodes the
spread of the density, fitting a wide kernel to volume-like regions and a tight
kernel to surface-like regions. We then extract an explicit mesh of a narrow
band around the surface, with width determined by the kernel size, and
fine-tune the radiance field within this band. At inference time, we cast rays
against the mesh and evaluate the radiance field only within the enclosed
region, greatly reducing the number of samples required. Experiments show that
our approach enables efficient rendering at very high fidelity. We also
demonstrate that the extracted envelope enables downstream applications such as
animation and simulation.
A method for neural radiance fields reduces rendering time while improving quality by smoothly transitioning from volumetric to surface-based rendering based on learned spatial density.
neural radiance fields volumetric formulation novel view synthesis volumetric encodings stochastic optimization surface-based rendering mesh envelope density spatially-varying kernel size radiance field inference animation simulation