S4: Self-Supervised Sensing Across the Spectrum
<|reference_start|>S4: Self-Supervised Sensing Across the Spectrum: Satellite image time series (SITS) segmentation is crucial for many applications like environmental monitoring, land cover mapping and agricultural crop type classification. However, training models for SITS segmentation remains a challenging task due to the lack of abundant training data, which requires fine grained annotation. We propose S4 a new self-supervised pre-training approach that significantly reduces the requirement for labeled training data by utilizing two new insights: (a) Satellites capture images in different parts of the spectrum such as radio frequencies, and visible frequencies. (b) Satellite imagery is geo-registered allowing for fine-grained spatial alignment. We use these insights to formulate pre-training tasks in S4. We also curate m2s2-SITS, a large-scale dataset of unlabeled, spatially-aligned, multi-modal and geographic specific SITS that serves as representative pre-training data for S4. Finally, we evaluate S4 on multiple SITS segmentation datasets and demonstrate its efficacy against competing baselines while using limited labeled data.<|reference_end|>
@article{shenoy2024s4:,
    title={S4: Self-Supervised Sensing Across the Spectrum},
    author={Jayanth Shenoy, Xingjian Davis Zhang, Shlok Mehrotra, Bill Tao, Rem
  Yang, Han Zhao, Deepak Vasisht},
    journal={arXiv preprint arXiv:2405.01656},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.01656},
    primaryClass={cs.CV cs.LG}
}