SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes
We present SpectroMotion, a novel approach that combines 3D Gaussian
Splatting (3DGS) with physically-based rendering (PBR) and deformation fields
to reconstruct dynamic specular scenes. Previous methods extending 3DGS to
model dynamic scenes have struggled to accurately represent specular surfaces.
Our method addresses this limitation by introducing a residual correction
technique for accurate surface normal computation during deformation,
complemented by a deformable environment map that adapts to time-varying
lighting conditions. We implement a coarse-to-fine training strategy that
significantly enhances both scene geometry and specular color prediction. We
demonstrate that our model outperforms prior methods for view synthesis of
scenes containing dynamic specular objects and that it is the only existing
3DGS method capable of synthesizing photorealistic real-world dynamic specular
scenes, outperforming state-of-the-art methods in rendering complex, dynamic,
and specular scenes.
6718937bb954d4a1d5cb9b2f
Cheng-De Fan
6718937bb954d4a1d5cb9b30
Chen-Wei Chang
admin_assigned
2024-10-23T07:44:17.309Z
63f4f85371a5d395c71d89d7
https://cdn-avatars.huggingface.co/v1/production/uploads/63f4f85371a5d395c71d89d7/VvGa9k3Rao3lAGPhMLiHJ.jpeg
Steven Chang
user
stevenchang
6718937bb954d4a1d5cb9b31
Yi-Ruei Liu
6718937bb954d4a1d5cb9b32
Jie-Ying Lee
claimed_verified
2024-10-23T15:06:51.172Z
655f1770f74fa124d1172ec1
/avatars/e4413693c34974fac75a438ffe2cc630.svg
Jay Lee
user
jayinnn
6718937bb954d4a1d5cb9b33
Jiun-Long Huang
6718937bb954d4a1d5cb9b34
Yu-Chee Tseng
6718937bb954d4a1d5cb9b35
Yu-Lun Liu
admin_assigned
2024-10-23T07:45:03.092Z
6459d5da3b6fafd9664807ab
/avatars/57430d1bbde3a2fe5586e5fbcafb0e74.svg
Yu-Lun Liu
user
yulunliu
SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes
We present SpectroMotion, a novel approach that combines 3D Gaussian
Splatting (3DGS) with physically-based rendering (PBR) and deformation fields
to reconstruct dynamic specular scenes. Previous methods extending 3DGS to
model dynamic scenes have struggled to accurately represent specular surfaces.
Our method addresses this limitation by introducing a residual correction
technique for accurate surface normal computation during deformation,
complemented by a deformable environment map that adapts to time-varying
lighting conditions. We implement a coarse-to-fine training strategy that
significantly enhances both scene geometry and specular color prediction. We
demonstrate that our model outperforms prior methods for view synthesis of
scenes containing dynamic specular objects and that it is the only existing
3DGS method capable of synthesizing photorealistic real-world dynamic specular
scenes, outperforming state-of-the-art methods in rendering complex, dynamic,
and specular scenes.
SpectroMotion improves upon 3D Gaussian Splatting by integrating physical-based rendering and deformation fields, incorporating a residual correction technique and deformable environment map to accurately reconstruct and synthesize dynamic specular scenes.
3D Gaussian Splatting (3DGS) physically-based rendering (PBR) deformation fields surface normal computation deformable environment map view synthesis photorealistic real-world dynamic specular scenes