DeepLab Attention [47] 75.1
CRF-RNN (baseline) [10] 74.7
DeepLab WSSL [48] 73.9
DeepLab [12] 72.7
Table 3: Mean Intersection over Union (IoU) results on PASCAL Context validation set
compared to other current methods.
Method Ours BoxSup [35] ParseNet [49] CRF-RNN [10] FCN-8s [3] CFM [28]
Mean IoU (%) 41.3 40.5 40.4 39.3 37.8 34.4
The output of the ﬁrst stage is fed into our CRF inference network. This is implemented using the mean ﬁeld update operations and their differentials described in
Section 5. Five iterations of mean ﬁeld inference were performed during training. Our
CRF network has two additional inputs in addition to segmentation unaries obtained from
the FCN-8s network: data from the object detector and superpixel oversegmentations of
the image.
We used the publicly available code and model of the Faster R-CNN [37] object detector. The fully automated version of GrabCut [38] was then used to obtain foregrounds
from the detection bounding boxes. These choices were made after conducting preliminary experiments with alternate detection and foreground segmentation algorithms.
We address the problem of semantic segmentation using deep learning. Most
segmentation systems include a Conditional Random Field (CRF) to produce a
structured output that is consistent with the image's visual features. Recent
deep learning approaches have incorporated CRFs into Convolutional Neural
Networks (CNNs), with some even training the CRF end-to-end with the rest of
the network. However, these approaches have not employed higher order
potentials, which have previously been shown to significantly improve
segmentation performance. In this paper, we demonstrate that two types of
higher order potential, based on object detections and superpixels, can be
included in a CRF embedded within a deep network. We design these higher order
potentials to allow inference with the differentiable mean field algorithm. As
a result, all the parameters of our richer CRF model can be learned end-to-end
with our pixelwise CNN classifier. We achieve state-of-the-art segmentation
performance on the PASCAL VOC benchmark with these trainable higher order
potentials.
Anurag Arnab Sadeep Jayasumana Shuai Zheng Philip Torr