Aerial Lifting: Neural Urban Semantic and Building Instance Lifting from Aerial Imagery
<|reference_start|>Aerial Lifting: Neural Urban Semantic and Building Instance Lifting from Aerial Imagery: We present a neural radiance field method for urban-scale semantic and building-level instance segmentation from aerial images by lifting noisy 2D labels to 3D. This is a challenging problem due to two primary reasons. Firstly, objects in urban aerial images exhibit substantial variations in size, including buildings, cars, and roads, which pose a significant challenge for accurate 2D segmentation. Secondly, the 2D labels generated by existing segmentation methods suffer from the multi-view inconsistency problem, especially in the case of aerial images, where each image captures only a small portion of the entire scene. To overcome these limitations, we first introduce a scale-adaptive semantic label fusion strategy that enhances the segmentation of objects of varying sizes by combining labels predicted from different altitudes, harnessing the novel-view synthesis capabilities of NeRF. We then introduce a novel cross-view instance label grouping strategy based on the 3D scene representation to mitigate the multi-view inconsistency problem in the 2D instance labels. Furthermore, we exploit multi-view reconstructed depth priors to improve the geometric quality of the reconstructed radiance field, resulting in enhanced segmentation results. Experiments on multiple real-world urban-scale datasets demonstrate that our approach outperforms existing methods, highlighting its effectiveness.<|reference_end|>
@article{zhang2024aerial,
    title={Aerial Lifting: Neural Urban Semantic and Building Instance Lifting from
  Aerial Imagery},
    author={Yuqi Zhang, Guanying Chen, Jiaxing Chen, and Shuguang Cui},
    journal={arXiv preprint arXiv:2403.11812},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.11812},
    primaryClass={cs.CV}
}