HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction
<|reference_start|>HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction: Reconstructing 3D scenes from multiple viewpoints is a fundamental task in stereo vision. Recently, advances in generalizable 3D Gaussian Splatting have enabled high-quality novel view synthesis for unseen scenes from sparse input views by feed-forward predicting per-pixel Gaussian parameters without extra optimization. However, existing methods typically generate single-scale 3D Gaussians, which lack representation of both large-scale structure and texture details, resulting in mislocation and artefacts. In this paper, we propose a novel framework, HiSplat, which introduces a hierarchical manner in generalizable 3D Gaussian Splatting to construct hierarchical 3D Gaussians via a coarse-to-fine strategy. Specifically, HiSplat generates large coarse-grained Gaussians to capture large-scale structures, followed by fine-grained Gaussians to enhance delicate texture details. To promote inter-scale interactions, we propose an Error Aware Module for Gaussian compensation and a Modulating Fusion Module for Gaussian repair. Our method achieves joint optimization of hierarchical representations, allowing for novel view synthesis using only two-view reference images. Comprehensive experiments on various datasets demonstrate that HiSplat significantly enhances reconstruction quality and cross-dataset generalization compared to prior single-scale methods. The corresponding ablation study and analysis of different-scale 3D Gaussians reveal the mechanism behind the effectiveness. Project website: https://open3dvlab.github.io/HiSplat/<|reference_end|>
@article{tang2024hisplat:,
    title={HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable
  Sparse-View Reconstruction},
    author={Shengji Tang, Weicai Ye, Peng Ye, Weihao Lin, Yang Zhou, Tao Chen,
  Wanli Ouyang},
    journal={arXiv preprint arXiv:2410.06245},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.06245},
    primaryClass={cs.CV}
}