Subsurface Scattering for 3D Gaussian Splatting
3D reconstruction and relighting of objects made from scattering materials
present a significant challenge due to the complex light transport beneath the
surface. 3D Gaussian Splatting introduced high-quality novel view synthesis at
real-time speeds. While 3D Gaussians efficiently approximate an object's
surface, they fail to capture the volumetric properties of subsurface
scattering. We propose a framework for optimizing an object's shape together
with the radiance transfer field given multi-view OLAT (one light at a time)
data. Our method decomposes the scene into an explicit surface represented as
3D Gaussians, with a spatially varying BRDF, and an implicit volumetric
representation of the scattering component. A learned incident light field
accounts for shadowing. We optimize all parameters jointly via ray-traced
differentiable rendering. Our approach enables material editing, relighting and
novel view synthesis at interactive rates. We show successful application on
synthetic data and introduce a newly acquired multi-view multi-light dataset of
objects in a light-stage setup. Compared to previous work we achieve comparable
or better results at a fraction of optimization and rendering time while
enabling detailed control over material attributes. Project page
https://sss.jdihlmann.com/
66c7f97be51ba300920429e4
Jan-Niklas Dihlmann
claimed_verified
2024-08-26T07:15:39.142Z
631b6dbcbf1351ed2bd05be1
https://cdn-avatars.huggingface.co/v1/production/uploads/631b6dbcbf1351ed2bd05be1/_kzcDD4qCifsb-651_LmY.png
Jan-Niklas Dihlmann
user
JDihlmann
66c7f97be51ba300920429e5
Arjun Majumdar
66c7f97be51ba300920429e6
Andreas Engelhardt
66c7f97be51ba300920429e7
Raphael Braun
66c7f97be51ba300920429e8
Hendrik P. A. Lensch
Subsurface Scattering for 3D Gaussian Splatting
3D reconstruction and relighting of objects made from scattering materials
present a significant challenge due to the complex light transport beneath the
surface. 3D Gaussian Splatting introduced high-quality novel view synthesis at
real-time speeds. While 3D Gaussians efficiently approximate an object's
surface, they fail to capture the volumetric properties of subsurface
scattering. We propose a framework for optimizing an object's shape together
with the radiance transfer field given multi-view OLAT (one light at a time)
data. Our method decomposes the scene into an explicit surface represented as
3D Gaussians, with a spatially varying BRDF, and an implicit volumetric
representation of the scattering component. A learned incident light field
accounts for shadowing. We optimize all parameters jointly via ray-traced
differentiable rendering. Our approach enables material editing, relighting and
novel view synthesis at interactive rates. We show successful application on
synthetic data and introduce a newly acquired multi-view multi-light dataset of
objects in a light-stage setup. Compared to previous work we achieve comparable
or better results at a fraction of optimization and rendering time while
enabling detailed control over material attributes. Project page
https://sss.jdihlmann.com/
A framework optimizes object shape and radiance transfer for detailed 3D reconstruction and relighting of scattering materials using 3D Gaussians and implicit volumetric representation.
3D Gaussian Splatting 3D Gaussians BRDF volumetric representation subsurface scattering incident light field ray-traced differentiable rendering material editing novel view synthesis light-stage setup