title: Adapting Vision Foundation Models for Robust Cloud Segmentation in
    Remote Sensing Images
summary: Cloud segmentation is a critical challenge in remote sensing image
  interpretation, as its accuracy directly impacts the effectiveness of
  subsequent data processing and analysis. Recently, vision foundation models
  (VFM) have demonstrated powerful generalization capabilities across various
  visual tasks. In this paper, we present a parameter-efficient adaptive
  approach, termed Cloud-Adapter, designed to enhance the accuracy and robustness
  of cloud segmentation. Our method leverages a VFM pretrained on general domain
  data, which remains frozen, eliminating the need for additional training.
  Cloud-Adapter incorporates a lightweight spatial perception module that
  initially utilizes a convolutional neural network (ConvNet) to extract dense
  spatial representations. These multi-scale features are then aggregated and
  serve as contextual inputs to an adapting module, which modulates the frozen
  transformer layers within the VFM. Experimental results demonstrate that the
  Cloud-Adapter approach, utilizing only 0.6% of the trainable parameters of the
  frozen backbone, achieves substantial performance gains. Cloud-Adapter
  consistently attains state-of-the-art (SOTA) performance across a wide variety
  of cloud segmentation datasets from multiple satellite sources, sensor series,
  data processing levels, land cover scenarios, and annotation granularities. We
  have released the source code and pretrained models at
  https://github.com/XavierJiezou/Cloud-Adapter to support further research.
paper_authors._id: 67416de111bf5a76d6eb47ca
paper_authors.name: Xuechao Zou
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-11-23T05:53:38.924Z
paper_authors._id: 6617af2beab5eef6b1e8bb9e
paper_authors.avatarUrl: /avatars/d939c02027916331d4c44119565f2ca6.svg
paper_authors.fullname: XavierJiezou
paper_authors.type: user
paper_authors.user: XavierJiezou
paper_authors._id: 67416de111bf5a76d6eb47cb
paper_authors.name: Shun Zhang
paper_authors._id: 67416de111bf5a76d6eb47cc
paper_authors.name: Kai Li
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-11-23T05:53:38.924Z
paper_authors._id: 6387676c23da90491eb9fb16
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1669818175965-noauth.jpeg
paper_authors.fullname: Kai Li
paper_authors.type: user
paper_authors.user: JusperLee
paper_authors._id: 67416de111bf5a76d6eb47cd
paper_authors.name: Shiying Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-25T20:58:29.826Z
paper_authors._id: 64b519944e5902e2fdfbde6b
paper_authors.avatarUrl: /avatars/e40f2cbd6ff1deec253016ddf76b2117.svg
paper_authors.fullname: wangshiying
paper_authors.type: user
paper_authors.user: shiyingstone
paper_authors._id: 67416de111bf5a76d6eb47ce
paper_authors.name: Junliang Xing
paper_authors._id: 67416de111bf5a76d6eb47cf
paper_authors.name: Lei Jin
paper_authors._id: 67416de111bf5a76d6eb47d0
paper_authors.name: Congyan Lang
paper_authors._id: 67416de111bf5a76d6eb47d1
paper_authors.name: Pin Tao
paper_title: Adapting Vision Foundation Models for Robust Cloud Segmentation in
    Remote Sensing Images
paper_summary: Cloud segmentation is a critical challenge in remote sensing image
  interpretation, as its accuracy directly impacts the effectiveness of
  subsequent data processing and analysis. Recently, vision foundation models
  (VFM) have demonstrated powerful generalization capabilities across various
  visual tasks. In this paper, we present a parameter-efficient adaptive
  approach, termed Cloud-Adapter, designed to enhance the accuracy and robustness
  of cloud segmentation. Our method leverages a VFM pretrained on general domain
  data, which remains frozen, eliminating the need for additional training.
  Cloud-Adapter incorporates a lightweight spatial perception module that
  initially utilizes a convolutional neural network (ConvNet) to extract dense
  spatial representations. These multi-scale features are then aggregated and
  serve as contextual inputs to an adapting module, which modulates the frozen
  transformer layers within the VFM. Experimental results demonstrate that the
  Cloud-Adapter approach, utilizing only 0.6% of the trainable parameters of the
  frozen backbone, achieves substantial performance gains. Cloud-Adapter
  consistently attains state-of-the-art (SOTA) performance across a wide variety
  of cloud segmentation datasets from multiple satellite sources, sensor series,
  data processing levels, land cover scenarios, and annotation granularities. We
  have released the source code and pretrained models at
  https://github.com/XavierJiezou/Cloud-Adapter to support further research.
paper_ai_summary: The Cloud-Adapter approach enhances cloud segmentation accuracy using a parameter-efficient method that adapts a pretrained vision foundation model with a lightweight spatial perception module.
paper_ai_keywords: vision foundation models (VFM) parameter-efficient Cloud-Adapter convolutional neural network (ConvNet) spatial perception module adapting module frozen transformer layers state-of-the-art (SOTA)