BAGS: Blur Agnostic Gaussian Splatting through Multi-Scale Kernel Modeling
<|reference_start|>BAGS: Blur Agnostic Gaussian Splatting through Multi-Scale Kernel Modeling: Recent efforts in using 3D Gaussians for scene reconstruction and novel view synthesis can achieve impressive results on curated benchmarks; however, images captured in real life are often blurry. In this work, we analyze the robustness of Gaussian-Splatting-based methods against various image blur, such as motion blur, defocus blur, downscaling blur, \etc. Under these degradations, Gaussian-Splatting-based methods tend to overfit and produce worse results than Neural-Radiance-Field-based methods. To address this issue, we propose Blur Agnostic Gaussian Splatting (BAGS). BAGS introduces additional 2D modeling capacities such that a 3D-consistent and high quality scene can be reconstructed despite image-wise blur. Specifically, we model blur by estimating per-pixel convolution kernels from a Blur Proposal Network (BPN). BPN is designed to consider spatial, color, and depth variations of the scene to maximize modeling capacity. Additionally, BPN also proposes a quality-assessing mask, which indicates regions where blur occur. Finally, we introduce a coarse-to-fine kernel optimization scheme; this optimization scheme is fast and avoids sub-optimal solutions due to a sparse point cloud initialization, which often occurs when we apply Structure-from-Motion on blurry images. We demonstrate that BAGS achieves photorealistic renderings under various challenging blur conditions and imaging geometry, while significantly improving upon existing approaches.<|reference_end|>
@article{peng2024bags:,
    title={BAGS: Blur Agnostic Gaussian Splatting through Multi-Scale Kernel
  Modeling},
    author={Cheng Peng, Yutao Tang, Yifan Zhou, Nengyu Wang, Xijun Liu, Deming Li,
  Rama Chellappa},
    journal={arXiv preprint arXiv:2403.04926},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.04926},
    primaryClass={cs.CV}
}