title: Transferable and Principled Efficiency for Open-Vocabulary Segmentation
summary: Recent success of pre-trained foundation vision-language models makes
  Open-Vocabulary Segmentation (OVS) possible. Despite the promising performance,
  this approach introduces heavy computational overheads for two challenges: 1)
  large model sizes of the backbone; 2) expensive costs during the fine-tuning.
  These challenges hinder this OVS strategy from being widely applicable and
  affordable in real-world scenarios. Although traditional methods such as model
  compression and efficient fine-tuning can address these challenges, they often
  rely on heuristics. This means that their solutions cannot be easily
  transferred and necessitate re-training on different models, which comes at a
  cost. In the context of efficient OVS, we target achieving performance that is
  comparable to or even better than prior OVS works based on large
  vision-language foundation models, by utilizing smaller models that incur lower
  training costs. The core strategy is to make our efficiency principled and thus
  seamlessly transferable from one OVS framework to others without further
  customization. Comprehensive experiments on diverse OVS benchmarks demonstrate
  our superior trade-off between segmentation accuracy and computation costs over
  previous works. Our code is available on https://github.com/Xujxyang/OpenTrans
paper_authors._id: 66188bdfbfea2f9d4c9ce05e
paper_authors.name: Jingxuan Xu
paper_authors._id: 66188bdfbfea2f9d4c9ce05f
paper_authors.name: Wuyang Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-18T12:34:16.822Z
paper_authors._id: 6489d8f967e8a51448487cbb
paper_authors.avatarUrl: /avatars/731732fb3405542b95f1ce0bc13db059.svg
paper_authors.fullname: Wuyang Chen
paper_authors.type: user
paper_authors.user: chenwydj
paper_authors._id: 66188bdfbfea2f9d4c9ce060
paper_authors.name: Yao Zhao
paper_authors._id: 66188bdfbfea2f9d4c9ce061
paper_authors.name: Yunchao Wei
paper_title: Transferable and Principled Efficiency for Open-Vocabulary Segmentation
paper_summary: Recent success of pre-trained foundation vision-language models makes
  Open-Vocabulary Segmentation (OVS) possible. Despite the promising performance,
  this approach introduces heavy computational overheads for two challenges: 1)
  large model sizes of the backbone; 2) expensive costs during the fine-tuning.
  These challenges hinder this OVS strategy from being widely applicable and
  affordable in real-world scenarios. Although traditional methods such as model
  compression and efficient fine-tuning can address these challenges, they often
  rely on heuristics. This means that their solutions cannot be easily
  transferred and necessitate re-training on different models, which comes at a
  cost. In the context of efficient OVS, we target achieving performance that is
  comparable to or even better than prior OVS works based on large
  vision-language foundation models, by utilizing smaller models that incur lower
  training costs. The core strategy is to make our efficiency principled and thus
  seamlessly transferable from one OVS framework to others without further
  customization. Comprehensive experiments on diverse OVS benchmarks demonstrate
  our superior trade-off between segmentation accuracy and computation costs over
  previous works. Our code is available on https://github.com/Xujxyang/OpenTrans
paper_ai_summary: The approach leverages smaller vision-language models to achieve efficient Open-Vocabulary Segmentation with comparable or better performance and lower computational costs compared to larger models.
paper_ai_keywords: pre-trained foundation vision-language models Open-Vocabulary Segmentation large model sizes fine-tuning model compression efficient fine-tuning efficiency OVS benchmarks segmentation accuracy computation costs