title: Progressive Language-guided Visual Learning for Multi-Task Visual
    Grounding
summary: Multi-task visual grounding (MTVG) includes two sub-tasks, i.e., Referring
  Expression Comprehension (REC) and Referring Expression Segmentation (RES). The
  existing representative approaches generally follow the research pipeline which
  mainly consists of three core procedures, including independent feature
  extraction for visual and linguistic modalities, respectively, cross-modal
  interaction module, and independent prediction heads for different sub-tasks.
  Albeit achieving remarkable performance, this research line has two
  limitations: 1) The linguistic content has not been fully injected into the
  entire visual backbone for boosting more effective visual feature extraction
  and it needs an extra cross-modal interaction module; 2) The relationship
  between REC and RES tasks is not effectively exploited to help the
  collaborative prediction for more accurate output. To deal with these problems,
  in this paper, we propose a Progressive Language-guided Visual Learning
  framework for multi-task visual grounding, called PLVL, which not only finely
  mine the inherent feature expression of the visual modality itself but also
  progressively inject the language information to help learn linguistic-related
  visual features. In this manner, our PLVL does not need additional cross-modal
  fusion module while fully introducing the language guidance. Furthermore, we
  analyze that the localization center for REC would help identify the
  to-be-segmented object region for RES to some extent. Inspired by this
  investigation, we design a multi-task head to accomplish collaborative
  predictions for these two sub-tasks. Extensive experiments conducted on several
  benchmark datasets comprehensively substantiate that our PLVL obviously
  outperforms the representative methods in both REC and RES tasks.
  https://github.com/jcwang0602/PLVL
paper_authors._id: 680a164533e15279df86adec
paper_authors.name: Jingchao Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-24T11:25:46.213Z
paper_authors._id: 65eaa07cb6c760d77468b4b6
paper_authors.avatarUrl: /avatars/4a1aae58986b40444351e0a167ca807c.svg
paper_authors.fullname: Jingchao Wang
paper_authors.type: user
paper_authors.user: jcwang0602
paper_authors._id: 680a164533e15279df86aded
paper_authors.name: Hong Wang
paper_authors._id: 680a164533e15279df86adee
paper_authors.name: Wenlong Zhang
paper_authors._id: 680a164533e15279df86adef
paper_authors.name: Kunhua Ji
paper_authors._id: 680a164533e15279df86adf0
paper_authors.name: Dingjiang Huang
paper_authors._id: 680a164533e15279df86adf1
paper_authors.name: Yefeng Zheng
paper_title: Progressive Language-guided Visual Learning for Multi-Task Visual
    Grounding
paper_summary: Multi-task visual grounding (MTVG) includes two sub-tasks, i.e., Referring
  Expression Comprehension (REC) and Referring Expression Segmentation (RES). The
  existing representative approaches generally follow the research pipeline which
  mainly consists of three core procedures, including independent feature
  extraction for visual and linguistic modalities, respectively, cross-modal
  interaction module, and independent prediction heads for different sub-tasks.
  Albeit achieving remarkable performance, this research line has two
  limitations: 1) The linguistic content has not been fully injected into the
  entire visual backbone for boosting more effective visual feature extraction
  and it needs an extra cross-modal interaction module; 2) The relationship
  between REC and RES tasks is not effectively exploited to help the
  collaborative prediction for more accurate output. To deal with these problems,
  in this paper, we propose a Progressive Language-guided Visual Learning
  framework for multi-task visual grounding, called PLVL, which not only finely
  mine the inherent feature expression of the visual modality itself but also
  progressively inject the language information to help learn linguistic-related
  visual features. In this manner, our PLVL does not need additional cross-modal
  fusion module while fully introducing the language guidance. Furthermore, we
  analyze that the localization center for REC would help identify the
  to-be-segmented object region for RES to some extent. Inspired by this
  investigation, we design a multi-task head to accomplish collaborative
  predictions for these two sub-tasks. Extensive experiments conducted on several
  benchmark datasets comprehensively substantiate that our PLVL obviously
  outperforms the representative methods in both REC and RES tasks.
  https://github.com/jcwang0602/PLVL
paper_ai_summary: A Progressive Language-guided Visual Learning framework for multi-task visual grounding that integrates linguistic information into the visual backbone and leverages shared features between sub-tasks to improve performance in both Referring Expression Comprehension and Referring Expression Segmentation.
paper_ai_keywords: Multi-task visual grounding Referring Expression Comprehension Referring Expression Segmentation independent feature extraction cross-modal interaction module independent prediction heads visual backbone language information linguistic-related visual features multi-task head shared features benchmark datasets