  Image-level weakly supervised semantic segmentation is a challenging problem
that has been deeply studied in recent years. Most of advanced solutions
exploit class activation map (CAM). However, CAMs can hardly serve as the
object mask due to the gap between full and weak supervisions. In this paper,
we propose a self-supervised equivariant attention mechanism (SEAM) to discover
additional supervision and narrow the gap. Our method is based on the
observation that equivariance is an implicit constraint in fully supervised
semantic segmentation, whose pixel-level labels take the same spatial
transformation as the input images during data augmentation. However, this
constraint is lost on the CAMs trained by image-level supervision. Therefore,
we propose consistency regularization on predicted CAMs from various
transformed images to provide self-supervision for network learning. Moreover,
we propose a pixel correlation module (PCM), which exploits context appearance
information and refines the prediction of current pixel by its similar
neighbors, leading to further improvement on CAMs consistency. Extensive
experiments on PASCAL VOC 2012 dataset demonstrate our method outperforms
state-of-the-art methods using the same level of supervision. The code is
released online.

Computer Vision and Pattern Recognition