Splatfacto-W: A Nerfstudio Implementation of Gaussian Splatting for
  Unconstrained Photo Collections
Novel view synthesis from unconstrained in-the-wild image collections remains
a significant yet challenging task due to photometric variations and transient
occluders that complicate accurate scene reconstruction. Previous methods have
approached these issues by integrating per-image appearance features embeddings
in Neural Radiance Fields (NeRFs). Although 3D Gaussian Splatting (3DGS) offers
faster training and real-time rendering, adapting it for unconstrained image
collections is non-trivial due to the substantially different architecture. In
this paper, we introduce Splatfacto-W, an approach that integrates per-Gaussian
neural color features and per-image appearance embeddings into the
rasterization process, along with a spherical harmonics-based background model
to represent varying photometric appearances and better depict backgrounds. Our
key contributions include latent appearance modeling, efficient transient
object handling, and precise background modeling. Splatfacto-W delivers
high-quality, real-time novel view synthesis with improved scene consistency in
in-the-wild scenarios. Our method improves the Peak Signal-to-Noise Ratio
(PSNR) by an average of 5.3 dB compared to 3DGS, enhances training speed by 150
times compared to NeRF-based methods, and achieves a similar rendering speed to
3DGS. Additional video results and code integrated into Nerfstudio are
available at https://kevinxu02.github.io/splatfactow/.
66987aae2ca9a02afe5c41bd
Congrong Xu
66987aae2ca9a02afe5c41be
Justin Kerr
66987aae2ca9a02afe5c41bf
Angjoo Kanazawa
Splatfacto-W: A Nerfstudio Implementation of Gaussian Splatting for
  Unconstrained Photo Collections
Novel view synthesis from unconstrained in-the-wild image collections remains
a significant yet challenging task due to photometric variations and transient
occluders that complicate accurate scene reconstruction. Previous methods have
approached these issues by integrating per-image appearance features embeddings
in Neural Radiance Fields (NeRFs). Although 3D Gaussian Splatting (3DGS) offers
faster training and real-time rendering, adapting it for unconstrained image
collections is non-trivial due to the substantially different architecture. In
this paper, we introduce Splatfacto-W, an approach that integrates per-Gaussian
neural color features and per-image appearance embeddings into the
rasterization process, along with a spherical harmonics-based background model
to represent varying photometric appearances and better depict backgrounds. Our
key contributions include latent appearance modeling, efficient transient
object handling, and precise background modeling. Splatfacto-W delivers
high-quality, real-time novel view synthesis with improved scene consistency in
in-the-wild scenarios. Our method improves the Peak Signal-to-Noise Ratio
(PSNR) by an average of 5.3 dB compared to 3DGS, enhances training speed by 150
times compared to NeRF-based methods, and achieves a similar rendering speed to
3DGS. Additional video results and code integrated into Nerfstudio are
available at https://kevinxu02.github.io/splatfactow/.
Splatfacto-W combines Gaussian neural color features and appearance embeddings with a spherical harmonics background model for real-time, high-quality view synthesis in unconstrained image collections, improving PSNR and training efficiency.
Neural Radiance Fields 3D Gaussian Splatting per-Gaussian neural color features per-image appearance embeddings spherical harmonics background model latent appearance modeling transient object handling precise background modeling