Spectral-GS: Taming 3D Gaussian Splatting with Spectral Entropy
<|reference_start|>Spectral-GS: Taming 3D Gaussian Splatting with Spectral Entropy: Recently, 3D Gaussian Splatting (3D-GS) has achieved impressive results in novel view synthesis, demonstrating high fidelity and efficiency. However, it easily exhibits needle-like artifacts, especially when increasing the sampling rate. Mip-Splatting tries to remove these artifacts with a 3D smoothing filter for frequency constraints and a 2D Mip filter for approximated supersampling. Unfortunately, it tends to produce over-blurred results, and sometimes needle-like Gaussians still persist. Our spectral analysis of the covariance matrix during optimization and densification reveals that current 3D-GS lacks shape awareness, relying instead on spectral radius and view positional gradients to determine splitting. As a result, needle-like Gaussians with small positional gradients and low spectral entropy fail to split and overfit high-frequency details. Furthermore, both the filters used in 3D-GS and Mip-Splatting reduce the spectral entropy and increase the condition number during zooming in to synthesize novel view, causing view inconsistencies and more pronounced artifacts. Our Spectral-GS, based on spectral analysis, introduces 3D shape-aware splitting and 2D view-consistent filtering strategies, effectively addressing these issues, enhancing 3D-GS's capability to represent high-frequency details without noticeable artifacts, and achieving high-quality photorealistic rendering.<|reference_end|>
@article{huang2024spectral-gs:,
    title={Spectral-GS: Taming 3D Gaussian Splatting with Spectral Entropy},
    author={Letian Huang and Jie Guo and Jialin Dan and Ruoyu Fu and Shujie Wang
  and Yuanqi Li and Yanwen Guo},
    journal={arXiv preprint arXiv:2409.12771},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.12771},
    primaryClass={cs.CV cs.GR}
}