title: INT: Instance-Specific Negative Mining for Task-Generic Promptable
    Segmentation
summary: Task-generic promptable image segmentation aims to achieve segmentation of
  diverse samples under a single task description by utilizing only one
  task-generic prompt. Current methods leverage the generalization capabilities
  of Vision-Language Models (VLMs) to infer instance-specific prompts from these
  task-generic prompts in order to guide the segmentation process. However, when
  VLMs struggle to generalise to some image instances, predicting
  instance-specific prompts becomes poor. To solve this problem, we introduce
  Instance-specific Negative Mining for Task-Generic
  Promptable Segmentation (INT). The key idea of INT is to adaptively
  reduce the influence of irrelevant (negative) prior knowledge whilst to
  increase the use the most plausible prior knowledge, selected by negative
  mining with higher contrast, in order to optimise instance-specific prompts
  generation. Specifically, INT consists of two components: (1) instance-specific
  prompt generation, which progressively fliters out incorrect information in
  prompt generation; (2) semantic mask generation, which ensures each image
  instance segmentation matches correctly the semantics of the instance-specific
  prompts. INT is validated on six datasets, including camouflaged objects and
  medical images, demonstrating its effectiveness, robustness and scalability.
paper_authors._id: 67a0a09da2d6613d77a7d10e
paper_authors.name: Jian Hu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-03T12:59:15.739Z
paper_authors._id: 65e1b6e9501590df0173cbd3
paper_authors.avatarUrl: /avatars/a73e2139700e23eff455734c99cef5ba.svg
paper_authors.fullname: Jian Hu
paper_authors.type: user
paper_authors.user: lwpyh
paper_authors._id: 67a0a09da2d6613d77a7d10f
paper_authors.name: Zixu Cheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-03T12:59:22.704Z
paper_authors._id: 667ee096b0fad0fdee319ed4
paper_authors.avatarUrl: /avatars/d9df687e8522d47f7fcefe40fd9b575b.svg
paper_authors.fullname: Zixu Cheng
paper_authors.type: user
paper_authors.user: Cade921
paper_authors._id: 67a0a09da2d6613d77a7d110
paper_authors.name: Shaogang Gong
paper_title: INT: Instance-Specific Negative Mining for Task-Generic Promptable
    Segmentation
paper_summary: Task-generic promptable image segmentation aims to achieve segmentation of
  diverse samples under a single task description by utilizing only one
  task-generic prompt. Current methods leverage the generalization capabilities
  of Vision-Language Models (VLMs) to infer instance-specific prompts from these
  task-generic prompts in order to guide the segmentation process. However, when
  VLMs struggle to generalise to some image instances, predicting
  instance-specific prompts becomes poor. To solve this problem, we introduce
  Instance-specific Negative Mining for Task-Generic
  Promptable Segmentation (INT). The key idea of INT is to adaptively
  reduce the influence of irrelevant (negative) prior knowledge whilst to
  increase the use the most plausible prior knowledge, selected by negative
  mining with higher contrast, in order to optimise instance-specific prompts
  generation. Specifically, INT consists of two components: (1) instance-specific
  prompt generation, which progressively fliters out incorrect information in
  prompt generation; (2) semantic mask generation, which ensures each image
  instance segmentation matches correctly the semantics of the instance-specific
  prompts. INT is validated on six datasets, including camouflaged objects and
  medical images, demonstrating its effectiveness, robustness and scalability.
paper_ai_keywords: Vision-Language Models (VLMs) instance-specific prompts task-generic prompts negative mining semantic mask generation camouflaged objects medical images