14 Arnab et al.
of the “front-end” module proposed in the Dilated Convolution Network (DCN) of [46]
instead of FCN-8s.
Table 4 shows that end-to-end training of the CRF yields considerable improvements
over piecewise training. This was the case when using either FCN-8s or DCN for
obtaining the initial unaries before performing CRF inference with higher order potentials.
This suggests that our CRF network module can be plugged into different architectures
and achieve performance improvements.
Baseline for detections To evaluate the efﬁcacy of our detection potentials, we formulate a simpler baseline since no other methods use detection information at inference
time (BoxSup [35] derives ground truth for training using ground-truth bounding boxes).
Our baseline is similar to CRF-RNN [10], but prior to CRF inference, we take the
segmentation mask from the object detection and add a unary potential proportional to
the detector’s conﬁdence to the unary potentials for those pixels. We then perform meanﬁeld inference (with only pairwise terms) on these “augmented” unaries. Using this
We address the problem of semantic segmentation using deep learning. Most
segmentation systems include a Conditional Random Field (CRF) to produce a
structured output that is consistent with the image's visual features. Recent
deep learning approaches have incorporated CRFs into Convolutional Neural
Networks (CNNs), with some even training the CRF end-to-end with the rest of
the network. However, these approaches have not employed higher order
potentials, which have previously been shown to significantly improve
segmentation performance. In this paper, we demonstrate that two types of
higher order potential, based on object detections and superpixels, can be
included in a CRF embedded within a deep network. We design these higher order
potentials to allow inference with the differentiable mean field algorithm. As
a result, all the parameters of our richer CRF model can be learned end-to-end
with our pixelwise CNN classifier. We achieve state-of-the-art segmentation
performance on the PASCAL VOC benchmark with these trainable higher order
potentials.
Anurag Arnab Sadeep Jayasumana Shuai Zheng Philip Torr