title: Going Denser with Open-Vocabulary Part Segmentation
summary: Object detection has been expanded from a limited number of categories to
  open vocabulary. Moving forward, a complete intelligent vision system requires
  understanding more fine-grained object descriptions, object parts. In this
  paper, we propose a detector with the ability to predict both open-vocabulary
  objects and their part segmentation. This ability comes from two designs.
  First, we train the detector on the joint of part-level, object-level and
  image-level data to build the multi-granularity alignment between language and
  image. Second, we parse the novel object into its parts by its dense semantic
  correspondence with the base object. These two designs enable the detector to
  largely benefit from various data sources and foundation models. In
  open-vocabulary part segmentation experiments, our method outperforms the
  baseline by 3.3sim7.3 mAP in cross-dataset generalization on PartImageNet,
  and improves the baseline by 7.3 novel AP_{50} in cross-category
  generalization on Pascal Part. Finally, we train a detector that generalizes to
  a wide range of part segmentation datasets while achieving better performance
  than dataset-specific training.
paper_authors._id: 646705fdcdea48159aa5e576
paper_authors.name: Peize Sun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-19T19:04:05.378Z
paper_authors._id: 640dc9bf8512ec51d7f0ac1a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/640dc9bf8512ec51d7f0ac1a/sT4rdEoQbzfW6D3xDVdqt.jpeg
paper_authors.fullname: peizesun
paper_authors.type: user
paper_authors.user: peizesun
paper_authors._id: 646705fdcdea48159aa5e577
paper_authors.name: Shoufa Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-23T09:50:21.566Z
paper_authors._id: 6412a33900634c4fe9873652
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6412a33900634c4fe9873652/Nmn_yRA1gGD2VO1YbSOYF.jpeg
paper_authors.fullname: Shoufa Chen
paper_authors.type: user
paper_authors.user: ShoufaChen
paper_authors._id: 646705fdcdea48159aa5e578
paper_authors.name: Chenchen Zhu
paper_authors._id: 646705fdcdea48159aa5e579
paper_authors.name: Fanyi Xiao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-23T09:50:59.370Z
paper_authors._id: 641cf83645487810d136cb34
paper_authors.avatarUrl: /avatars/95bfc200f86abbe9e8df0308517a9ce4.svg
paper_authors.fullname: Fanyi Xiao
paper_authors.type: user
paper_authors.user: fanyix
paper_authors._id: 646705fdcdea48159aa5e57a
paper_authors.name: Ping Luo
paper_authors._id: 646705fdcdea48159aa5e57b
paper_authors.name: Saining Xie
paper_authors._id: 646705fdcdea48159aa5e57c
paper_authors.name: Zhicheng Yan
paper_title: Going Denser with Open-Vocabulary Part Segmentation
paper_summary: Object detection has been expanded from a limited number of categories to
  open vocabulary. Moving forward, a complete intelligent vision system requires
  understanding more fine-grained object descriptions, object parts. In this
  paper, we propose a detector with the ability to predict both open-vocabulary
  objects and their part segmentation. This ability comes from two designs.
  First, we train the detector on the joint of part-level, object-level and
  image-level data to build the multi-granularity alignment between language and
  image. Second, we parse the novel object into its parts by its dense semantic
  correspondence with the base object. These two designs enable the detector to
  largely benefit from various data sources and foundation models. In
  open-vocabulary part segmentation experiments, our method outperforms the
  baseline by 3.3sim7.3 mAP in cross-dataset generalization on PartImageNet,
  and improves the baseline by 7.3 novel AP_{50} in cross-category
  generalization on Pascal Part. Finally, we train a detector that generalizes to
  a wide range of part segmentation datasets while achieving better performance
  than dataset-specific training.
paper_ai_summary: A detector that predicts open-vocabulary objects and their parts using multi-granularity alignment outperforms baselines in cross-dataset and cross-category generalization.
paper_ai_keywords: open vocabulary object detection part segmentation multi-granularity alignment dense semantic correspondence cross-dataset generalization cross-category generalization dataset-specific training mAP novel AP$_{50}$ PartImageNet Pascal Part