title: SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes
summary: We present SpectroMotion, a novel approach that combines 3D Gaussian
  Splatting (3DGS) with physically-based rendering (PBR) and deformation fields
  to reconstruct dynamic specular scenes. Previous methods extending 3DGS to
  model dynamic scenes have struggled to accurately represent specular surfaces.
  Our method addresses this limitation by introducing a residual correction
  technique for accurate surface normal computation during deformation,
  complemented by a deformable environment map that adapts to time-varying
  lighting conditions. We implement a coarse-to-fine training strategy that
  significantly enhances both scene geometry and specular color prediction. We
  demonstrate that our model outperforms prior methods for view synthesis of
  scenes containing dynamic specular objects and that it is the only existing
  3DGS method capable of synthesizing photorealistic real-world dynamic specular
  scenes, outperforming state-of-the-art methods in rendering complex, dynamic,
  and specular scenes.
paper_authors._id: 6718937bb954d4a1d5cb9b2f
paper_authors.name: Cheng-De Fan
paper_authors._id: 6718937bb954d4a1d5cb9b30
paper_authors.name: Chen-Wei Chang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-23T07:44:17.309Z
paper_authors._id: 63f4f85371a5d395c71d89d7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63f4f85371a5d395c71d89d7/VvGa9k3Rao3lAGPhMLiHJ.jpeg
paper_authors.fullname: Steven Chang
paper_authors.type: user
paper_authors.user: stevenchang
paper_authors._id: 6718937bb954d4a1d5cb9b31
paper_authors.name: Yi-Ruei Liu
paper_authors._id: 6718937bb954d4a1d5cb9b32
paper_authors.name: Jie-Ying Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-23T15:06:51.172Z
paper_authors._id: 655f1770f74fa124d1172ec1
paper_authors.avatarUrl: /avatars/e4413693c34974fac75a438ffe2cc630.svg
paper_authors.fullname: Jay Lee
paper_authors.type: user
paper_authors.user: jayinnn
paper_authors._id: 6718937bb954d4a1d5cb9b33
paper_authors.name: Jiun-Long Huang
paper_authors._id: 6718937bb954d4a1d5cb9b34
paper_authors.name: Yu-Chee Tseng
paper_authors._id: 6718937bb954d4a1d5cb9b35
paper_authors.name: Yu-Lun Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-23T07:45:03.092Z
paper_authors._id: 6459d5da3b6fafd9664807ab
paper_authors.avatarUrl: /avatars/57430d1bbde3a2fe5586e5fbcafb0e74.svg
paper_authors.fullname: Yu-Lun Liu
paper_authors.type: user
paper_authors.user: yulunliu
paper_title: SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes
paper_summary: We present SpectroMotion, a novel approach that combines 3D Gaussian
  Splatting (3DGS) with physically-based rendering (PBR) and deformation fields
  to reconstruct dynamic specular scenes. Previous methods extending 3DGS to
  model dynamic scenes have struggled to accurately represent specular surfaces.
  Our method addresses this limitation by introducing a residual correction
  technique for accurate surface normal computation during deformation,
  complemented by a deformable environment map that adapts to time-varying
  lighting conditions. We implement a coarse-to-fine training strategy that
  significantly enhances both scene geometry and specular color prediction. We
  demonstrate that our model outperforms prior methods for view synthesis of
  scenes containing dynamic specular objects and that it is the only existing
  3DGS method capable of synthesizing photorealistic real-world dynamic specular
  scenes, outperforming state-of-the-art methods in rendering complex, dynamic,
  and specular scenes.
paper_ai_keywords: 3D Gaussian Splatting (3DGS) physically-based rendering (PBR) deformation fields specular surfaces residual correction technique surface normal computation deformable environment map coarse-to-fine training strategy view synthesis photorealistic real-world dynamic specular scenes rendering complex, dynamic, and specular scenes