Gaussian Time Machine: A Real-Time Rendering Methodology for Time-Variant Appearances
<|reference_start|>Gaussian Time Machine: A Real-Time Rendering Methodology for Time-Variant Appearances: Recent advancements in neural rendering techniques have significantly enhanced the fidelity of 3D reconstruction. Notably, the emergence of 3D Gaussian Splatting (3DGS) has marked a significant milestone by adopting a discrete scene representation, facilitating efficient training and real-time rendering. Several studies have successfully extended the real-time rendering capability of 3DGS to dynamic scenes. However, a challenge arises when training images are captured under vastly differing weather and lighting conditions. This scenario poses a challenge for 3DGS and its variants in achieving accurate reconstructions. Although NeRF-based methods (NeRF-W, CLNeRF) have shown promise in handling such challenging conditions, their computational demands hinder real-time rendering capabilities. In this paper, we present Gaussian Time Machine (GTM) which models the time-dependent attributes of Gaussian primitives with discrete time embedding vectors decoded by a lightweight Multi-Layer-Perceptron(MLP). By adjusting the opacity of Gaussian primitives, we can reconstruct visibility changes of objects. We further propose a decomposed color model for improved geometric consistency. GTM achieved state-of-the-art rendering fidelity on 3 datasets and is 100 times faster than NeRF-based counterparts in rendering. Moreover, GTM successfully disentangles the appearance changes and renders smooth appearance interpolation.<|reference_end|>
@article{shen2024gaussian,
    title={Gaussian Time Machine: A Real-Time Rendering Methodology for
  Time-Variant Appearances},
    author={Licheng Shen, Ho Ngai Chow, Lingyun Wang, Tong Zhang, Mengqiu Wang,
  Yuxing Han},
    journal={arXiv preprint arXiv:2405.13694},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.13694},
    primaryClass={cs.CV}
}