Pushing the Limits of Vision-Language Models in Remote Sensing without Human Annotations
<|reference_start|>Pushing the Limits of Vision-Language Models in Remote Sensing without Human Annotations: The prominence of generalized foundation models in vision-language integration has witnessed a surge, given their multifarious applications. Within the natural domain, the procurement of vision-language datasets to construct these foundation models is facilitated by their abundant availability and the ease of web crawling. Conversely, in the remote sensing domain, although vision-language datasets exist, their volume is suboptimal for constructing robust foundation models. This study introduces an approach to curate vision-language datasets by employing an image decoding machine learning model, negating the need for human-annotated labels. Utilizing this methodology, we amassed approximately 9.6 million vision-language paired datasets in VHR imagery. The resultant model outperformed counterparts that did not leverage publicly available vision-language datasets, particularly in downstream tasks such as zero-shot classification, semantic localization, and image-text retrieval. Moreover, in tasks exclusively employing vision encoders, such as linear probing and k-NN classification, our model demonstrated superior efficacy compared to those relying on domain-specific vision-language datasets.<|reference_end|>
@article{cha2024pushing,
    title={Pushing the Limits of Vision-Language Models in Remote Sensing without
  Human Annotations},
    author={Keumgang Cha, Donggeun Yu, and Junghoon Seo},
    journal={arXiv preprint arXiv:2409.07048},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.07048},
    primaryClass={cs.CV}
}