title: CosmoCLIP: Generalizing Large Vision-Language Models for Astronomical
    Imaging
summary: Existing vision-text contrastive learning models enhance representation
  transferability and support zero-shot prediction by matching paired image and
  caption embeddings while pushing unrelated pairs apart. However, astronomical
  image-label datasets are significantly smaller compared to general image and
  label datasets available from the internet. We introduce CosmoCLIP, an
  astronomical image-text contrastive learning framework precisely fine-tuned on
  the pre-trained CLIP model using SpaceNet and BLIP-based captions. SpaceNet,
  attained via FLARE, constitutes ~13k optimally distributed images, while BLIP
  acts as a rich knowledge extractor. The rich semantics derived from this
  SpaceNet and BLIP descriptions, when learned contrastively, enable CosmoCLIP to
  achieve superior generalization across various in-domain and out-of-domain
  tasks. Our results demonstrate that CosmoCLIP is a straightforward yet powerful
  framework, significantly outperforming CLIP in zero-shot classification and
  image-text retrieval tasks.
paper_authors._id: 668fb090c9ca04b1c10d53a9
paper_authors.name: Raza Imam
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-11T10:58:12.875Z
paper_authors._id: 664396275778efc851b14408
paper_authors.avatarUrl: /avatars/8ea99f8c6c59a7d49295aebfea026963.svg
paper_authors.fullname: Raza Imam
paper_authors.type: user
paper_authors.user: razaimam45
paper_authors._id: 668fb090c9ca04b1c10d53aa
paper_authors.name: Mohammed Talha Alam
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-11T10:58:19.295Z
paper_authors._id: 65a942038b3bd440586ae467
paper_authors.avatarUrl: /avatars/5c109e67f66bd47a2ea85db837e49e6e.svg
paper_authors.fullname: Mohammed Talha Alam
paper_authors.type: user
paper_authors.user: talha-alam
paper_authors._id: 668fb090c9ca04b1c10d53ab
paper_authors.name: Umaima Rahman
paper_authors._id: 668fb090c9ca04b1c10d53ac
paper_authors.name: Mohsen Guizani
paper_authors._id: 668fb090c9ca04b1c10d53ad
paper_authors.name: Fakhri Karray
paper_title: CosmoCLIP: Generalizing Large Vision-Language Models for Astronomical
    Imaging
paper_summary: Existing vision-text contrastive learning models enhance representation
  transferability and support zero-shot prediction by matching paired image and
  caption embeddings while pushing unrelated pairs apart. However, astronomical
  image-label datasets are significantly smaller compared to general image and
  label datasets available from the internet. We introduce CosmoCLIP, an
  astronomical image-text contrastive learning framework precisely fine-tuned on
  the pre-trained CLIP model using SpaceNet and BLIP-based captions. SpaceNet,
  attained via FLARE, constitutes ~13k optimally distributed images, while BLIP
  acts as a rich knowledge extractor. The rich semantics derived from this
  SpaceNet and BLIP descriptions, when learned contrastively, enable CosmoCLIP to
  achieve superior generalization across various in-domain and out-of-domain
  tasks. Our results demonstrate that CosmoCLIP is a straightforward yet powerful
  framework, significantly outperforming CLIP in zero-shot classification and
  image-text retrieval tasks.
paper_ai_summary: CosmoCLIP enhances astronomical image-text contrastive learning by fine-tuning the pre-trained CLIP model using SpaceNet and BLIP captions, achieving superior generalization and performance in zero-shot classification and image-text retrieval.
paper_ai_keywords: contrastive learning CLIP model SpaceNet BLIP FLARE zero-shot classification image-text retrieval