title: DOEI: Dual Optimization of Embedding Information for Attention-Enhanced
    Class Activation Maps
summary: Weakly supervised semantic segmentation (WSSS) typically utilizes limited
  semantic annotations to obtain initial Class Activation Maps (CAMs). However,
  due to the inadequate coupling between class activation responses and semantic
  information in high-dimensional space, the CAM is prone to object co-occurrence
  or under-activation, resulting in inferior recognition accuracy. To tackle this
  issue, we propose DOEI, Dual Optimization of Embedding Information, a novel
  approach that reconstructs embedding representations through semantic-aware
  attention weight matrices to optimize the expression capability of embedding
  information. Specifically, DOEI amplifies tokens with high confidence and
  suppresses those with low confidence during the class-to-patch interaction.
  This alignment of activation responses with semantic information strengthens
  the propagation and decoupling of target features, enabling the generated
  embeddings to more accurately represent target features in high-level semantic
  space. In addition, we propose a hybrid-feature alignment module in DOEI that
  combines RGB values, embedding-guided features, and self-attention weights to
  increase the reliability of candidate tokens. Comprehensive experiments show
  that DOEI is an effective plug-and-play module that empowers state-of-the-art
  visual transformer-based WSSS models to significantly improve the quality of
  CAMs and segmentation performance on popular benchmarks, including PASCAL VOC
  (+3.6%, +1.5%, +1.2% mIoU) and MS COCO (+1.2%, +1.6% mIoU). Code will be
  available at https://github.com/AIGeeksGroup/DOEI.
paper_authors._id: 67c05aeca2a76d8a27d33c8a
paper_authors.name: Hongjie Zhu
paper_authors._id: 67c05aeca2a76d8a27d33c8b
paper_authors.name: Zeyu Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T12:53:51.821Z
paper_authors._id: 64ec877bb93654d4ca5c92e9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64ec877bb93654d4ca5c92e9/GvHk_KSdE9Rhnk_o-NaZX.jpeg
paper_authors.fullname: Zeyu Zhang
paper_authors.type: user
paper_authors.user: SteveZeyuZhang
paper_authors._id: 67c05aeca2a76d8a27d33c8c
paper_authors.name: Guansong Pang
paper_authors._id: 67c05aeca2a76d8a27d33c8d
paper_authors.name: Xu Wang
paper_authors._id: 67c05aeca2a76d8a27d33c8e
paper_authors.name: Shimin Wen
paper_authors._id: 67c05aeca2a76d8a27d33c8f
paper_authors.name: Yu Bai
paper_authors._id: 67c05aeca2a76d8a27d33c90
paper_authors.name: Daji Ergu
paper_authors._id: 67c05aeca2a76d8a27d33c91
paper_authors.name: Ying Cai
paper_authors._id: 67c05aeca2a76d8a27d33c92
paper_authors.name: Yang Zhao
paper_title: DOEI: Dual Optimization of Embedding Information for Attention-Enhanced
    Class Activation Maps
paper_summary: Weakly supervised semantic segmentation (WSSS) typically utilizes limited
  semantic annotations to obtain initial Class Activation Maps (CAMs). However,
  due to the inadequate coupling between class activation responses and semantic
  information in high-dimensional space, the CAM is prone to object co-occurrence
  or under-activation, resulting in inferior recognition accuracy. To tackle this
  issue, we propose DOEI, Dual Optimization of Embedding Information, a novel
  approach that reconstructs embedding representations through semantic-aware
  attention weight matrices to optimize the expression capability of embedding
  information. Specifically, DOEI amplifies tokens with high confidence and
  suppresses those with low confidence during the class-to-patch interaction.
  This alignment of activation responses with semantic information strengthens
  the propagation and decoupling of target features, enabling the generated
  embeddings to more accurately represent target features in high-level semantic
  space. In addition, we propose a hybrid-feature alignment module in DOEI that
  combines RGB values, embedding-guided features, and self-attention weights to
  increase the reliability of candidate tokens. Comprehensive experiments show
  that DOEI is an effective plug-and-play module that empowers state-of-the-art
  visual transformer-based WSSS models to significantly improve the quality of
  CAMs and segmentation performance on popular benchmarks, including PASCAL VOC
  (+3.6%, +1.5%, +1.2% mIoU) and MS COCO (+1.2%, +1.6% mIoU). Code will be
  available at https://github.com/AIGeeksGroup/DOEI.
paper_ai_keywords: Class Activation Maps (CAMs) zero object co-occurrence under-activation Dual Optimization of Embedding Information (DOEI) semantic-aware optimal weight matrices tokens class-to-patch applications activation responses high-level semantic space hybrid-feature alignment module embedding-guided features self-attention weights visual transformer-based WSSS models mIoU (mean Intersection over Union)