SG-BEV: Satellite-Guided BEV Fusion for Cross-View Semantic Segmentation
<|reference_start|>SG-BEV: Satellite-Guided BEV Fusion for Cross-View Semantic Segmentation: This paper aims at achieving fine-grained building attribute segmentation in a cross-view scenario, i.e., using satellite and street-view image pairs. The main challenge lies in overcoming the significant perspective differences between street views and satellite views. In this work, we introduce SG-BEV, a novel approach for satellite-guided BEV fusion for cross-view semantic segmentation. To overcome the limitations of existing cross-view projection methods in capturing the complete building facade features, we innovatively incorporate Bird's Eye View (BEV) method to establish a spatially explicit mapping of street-view features. Moreover, we fully leverage the advantages of multiple perspectives by introducing a novel satellite-guided reprojection module, optimizing the uneven feature distribution issues associated with traditional BEV methods. Our method demonstrates significant improvements on four cross-view datasets collected from multiple cities, including New York, San Francisco, and Boston. On average across these datasets, our method achieves an increase in mIOU by 10.13% and 5.21% compared with the state-of-the-art satellite-based and cross-view methods. The code and datasets of this work will be released at https://github.com/yejy53/SG-BEV.<|reference_end|>
@article{ye2024sg-bev:,
    title={SG-BEV: Satellite-Guided BEV Fusion for Cross-View Semantic Segmentation},
    author={Junyan Ye, Qiyan Luo, Jinhua Yu, Huaping Zhong, Zhimeng Zheng, Conghui
  He, Weijia Li},
    journal={arXiv preprint arXiv:2404.02638},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.02638},
    primaryClass={cs.CV}
}