  We unveil a long-standing problem in the prevailing co-saliency detection
systems: there is indeed inconsistency between training and testing.
Constructing a high-quality co-saliency detection dataset involves
time-consuming and labor-intensive pixel-level labeling, which has forced most
recent works to rely instead on semantic segmentation or saliency detection
datasets for training. However, the lack of proper co-saliency and the absence
of multiple foreground objects in these datasets can lead to spurious
variations and inherent biases learned by models. To tackle this, we introduce
the idea of counterfactual training through context adjustment and propose a
"cost-free" group-cut-paste (GCP) procedure to leverage off-the-shelf images
and synthesize new samples. Following GCP, we collect a novel dataset called
Context Adjustment Training (CAT). CAT consists of 33,500 images, which is four
times larger than the current co-saliency detection datasets. All samples are
automatically annotated with high-quality mask annotations, object categories,
and edge maps. Extensive experiments on recent benchmarks are conducted, show
that CAT can improve various state-of-the-art models by a large margin (5% ~
25%). We hope that the scale, diversity, and quality of our dataset can benefit
researchers in this area and beyond. Our dataset will be publicly accessible
through our project page.

Computer Vision and Pattern Recognition