DF4LCZ: A SAM-Empowered Data Fusion Framework for Scene-Level Local Climate Zone Classification
<|reference_start|>DF4LCZ: A SAM-Empowered Data Fusion Framework for Scene-Level Local Climate Zone Classification: Recent advancements in remote sensing (RS) technologies have shown their potential in accurately classifying local climate zones (LCZs). However, traditional scene-level methods using convolutional neural networks (CNNs) often struggle to integrate prior knowledge of ground objects effectively. Moreover, commonly utilized data sources like Sentinel-2 encounter difficulties in capturing detailed ground object information. To tackle these challenges, we propose a data fusion method that integrates ground object priors extracted from high-resolution Google imagery with Sentinel-2 multispectral imagery. The proposed method introduces a novel Dual-stream Fusion framework for LCZ classification (DF4LCZ), integrating instance-based location features from Google imagery with the scene-level spatial-spectral features extracted from Sentinel-2 imagery. The framework incorporates a Graph Convolutional Network (GCN) module empowered by the Segment Anything Model (SAM) to enhance feature extraction from Google imagery. Simultaneously, the framework employs a 3D-CNN architecture to learn the spectral-spatial features of Sentinel-2 imagery. Experiments are conducted on a multi-source remote sensing image dataset specifically designed for LCZ classification, validating the effectiveness of the proposed DF4LCZ. The related code and dataset are available at https://github.com/ctrlovefly/DF4LCZ.<|reference_end|>
@article{wu2024df4lcz:,
    title={DF4LCZ: A SAM-Empowered Data Fusion Framework for Scene-Level Local
  Climate Zone Classification},
    author={Qianqian Wu, Xianping Ma, Jialu Sui, Man-On Pun},
    journal={arXiv preprint arXiv:2403.09367},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.09367},
    primaryClass={cs.CV}
}