title: INTRA: Interaction Relationship-aware Weakly Supervised Affordance
    Grounding
summary: Affordance denotes the potential interactions inherent in objects. The
  perception of affordance can enable intelligent agents to navigate and interact
  with new environments efficiently. Weakly supervised affordance grounding
  teaches agents the concept of affordance without costly pixel-level
  annotations, but with exocentric images. Although recent advances in weakly
  supervised affordance grounding yielded promising results, there remain
  challenges including the requirement for paired exocentric and egocentric image
  dataset, and the complexity in grounding diverse affordances for a single
  object. To address them, we propose INTeraction Relationship-aware weakly
  supervised Affordance grounding (INTRA). Unlike prior arts, INTRA recasts this
  problem as representation learning to identify unique features of interactions
  through contrastive learning with exocentric images only, eliminating the need
  for paired datasets. Moreover, we leverage vision-language model embeddings for
  performing affordance grounding flexibly with any text, designing
  text-conditioned affordance map generation to reflect interaction relationship
  for contrastive learning and enhancing robustness with our text synonym
  augmentation. Our method outperformed prior arts on diverse datasets such as
  AGD20K, IIT-AFF, CAD and UMD. Additionally, experimental results demonstrate
  that our method has remarkable domain scalability for synthesized images /
  illustrations and is capable of performing affordance grounding for novel
  interactions and objects.
paper_authors._id: 66e11334a533df3d825ffe22
paper_authors.name: Ji Ha Jang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-11T07:49:07.388Z
paper_authors._id: 64c77dc26a26cddbec858bcb
paper_authors.avatarUrl: /avatars/635e64f8205c194e63876cba9d8c5449.svg
paper_authors.fullname: jihajang
paper_authors.type: user
paper_authors.user: jeeit17
paper_authors._id: 66e11334a533df3d825ffe23
paper_authors.name: Hoigi Seo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-11T07:26:15.433Z
paper_authors._id: 633e6f07309a99325095dd42
paper_authors.avatarUrl: /avatars/57b91a488ac1745b3c0509c04eb6ad93.svg
paper_authors.fullname: Hoigi Seo
paper_authors.type: user
paper_authors.user: Agorium
paper_authors._id: 66e11334a533df3d825ffe24
paper_authors.name: Se Young Chun
paper_title: INTRA: Interaction Relationship-aware Weakly Supervised Affordance
    Grounding
paper_summary: Affordance denotes the potential interactions inherent in objects. The
  perception of affordance can enable intelligent agents to navigate and interact
  with new environments efficiently. Weakly supervised affordance grounding
  teaches agents the concept of affordance without costly pixel-level
  annotations, but with exocentric images. Although recent advances in weakly
  supervised affordance grounding yielded promising results, there remain
  challenges including the requirement for paired exocentric and egocentric image
  dataset, and the complexity in grounding diverse affordances for a single
  object. To address them, we propose INTeraction Relationship-aware weakly
  supervised Affordance grounding (INTRA). Unlike prior arts, INTRA recasts this
  problem as representation learning to identify unique features of interactions
  through contrastive learning with exocentric images only, eliminating the need
  for paired datasets. Moreover, we leverage vision-language model embeddings for
  performing affordance grounding flexibly with any text, designing
  text-conditioned affordance map generation to reflect interaction relationship
  for contrastive learning and enhancing robustness with our text synonym
  augmentation. Our method outperformed prior arts on diverse datasets such as
  AGD20K, IIT-AFF, CAD and UMD. Additionally, experimental results demonstrate
  that our method has remarkable domain scalability for synthesized images /
  illustrations and is capable of performing affordance grounding for novel
  interactions and objects.
paper_ai_keywords: weakly supervised affordance grounding exocentric images egocentric image dataset representation learning contrastive learning vision-language model embeddings text-conditioned affordance map generation text synonym augmentation domain scalability affordance grounding