Two Stream Encoder GAN With Progressive Training for Co Saliency Detection
The recent end to end co saliency models have good performance, however, they cannot express the semantic consistency among a group of images well and usually require many co saliency labels. To this end, a two stream encoder generative adversarial network (TSE GAN) with progressive training is proposed in this paper. In the pre training stage, the salient object detection generative adversarial networks (SOD GAN) and classification network (CN) are separately trained by the salient object detection (SOD) datasets and co saliency datasets with only category labels to learn the intra saliency and preliminary inter saliency cues and alleviate the problem of insufficient co saliency labels. In the second training stage, the backbone of TSE GAN is inherited from the trained SOD GAN, the encoder of trained SOD GAN (SOD Encoder) is used to extract intra saliency features, the group wise semantic encoder (GS Encoder) is constructed by the multi level group wise category features extracted from CN for extracting inter saliency features with better semantic consistency, the TSE GAN constructed by incorporating the GS Encoder into SOD GAN is trained on co saliency datasets for co saliency detection. The comprehensive comparisons with 13 state of the art methods demonstrate the effectiveness of proposed method.
IEEE Signal Processing Letters