RadSplat: Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPS
<|reference_start|>RadSplat: Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPS: Recent advances in view synthesis and real-time rendering have achieved photorealistic quality at impressive rendering speeds. While Radiance Field-based methods achieve state-of-the-art quality in challenging scenarios such as in-the-wild captures and large-scale scenes, they often suffer from excessively high compute requirements linked to volumetric rendering. Gaussian Splatting-based methods, on the other hand, rely on rasterization and naturally achieve real-time rendering but suffer from brittle optimization heuristics that underperform on more challenging scenes. In this work, we present RadSplat, a lightweight method for robust real-time rendering of complex scenes. Our main contributions are threefold. First, we use radiance fields as a prior and supervision signal for optimizing point-based scene representations, leading to improved quality and more robust optimization. Next, we develop a novel pruning technique reducing the overall point count while maintaining high quality, leading to smaller and more compact scene representations with faster inference speeds. Finally, we propose a novel test-time filtering approach that further accelerates rendering and allows to scale to larger, house-sized scenes. We find that our method enables state-of-the-art synthesis of complex captures at 900+ FPS.<|reference_end|>
@article{niemeyer2024radsplat:,
    title={RadSplat: Radiance Field-Informed Gaussian Splatting for Robust
  Real-Time Rendering with 900+ FPS},
    author={Michael Niemeyer, Fabian Manhardt, Marie-Julie Rakotosaona, Michael
  Oechsle, Daniel Duckworth, Rama Gosula, Keisuke Tateno, John Bates, Dominik
  Kaeser, Federico Tombari},
    journal={arXiv preprint arXiv:2403.13806},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.13806},
    primaryClass={cs.CV cs.GR}
}