nd/summationtextnd
i=1Q(X(i)
d=ld) ifb= 0,
wDetsd
nd/summationtextnd
i=1(1−Q(X(i)
d=ld))otherwise.
(7)
It is possible to increase the number of parameters in ψDet
d(.). Since we use backpropagation to learn these parameters automatically during end-to-end training, it is desirable
to have a high number of parameters to increase the ﬂexibility of the model. Following
10 Arnab et al.
this idea, we made the weight wDetclass speciﬁc, that is, a function wDet(ld)is used
instead ofwDetin Eqs. (2),(6)and (7). The underlying assumption is that detector
outputs can be very helpful for certain classes, while being not so useful for classes that
the detector performs poorly on, or classes for which foreground segmentation is often
inaccurate.
Note that due to the presence of detection potentials in the CRF, error differentials
calculated with respect to the Xvariable unary potentials and pairwise parameters will
no longer be valid in the forms described in [10]. The error differentials with respect to
We address the problem of semantic segmentation using deep learning. Most
segmentation systems include a Conditional Random Field (CRF) to produce a
structured output that is consistent with the image's visual features. Recent
deep learning approaches have incorporated CRFs into Convolutional Neural
Networks (CNNs), with some even training the CRF end-to-end with the rest of
the network. However, these approaches have not employed higher order
potentials, which have previously been shown to significantly improve
segmentation performance. In this paper, we demonstrate that two types of
higher order potential, based on object detections and superpixels, can be
included in a CRF embedded within a deep network. We design these higher order
potentials to allow inference with the differentiable mean field algorithm. As
a result, all the parameters of our richer CRF model can be learned end-to-end
with our pixelwise CNN classifier. We achieve state-of-the-art segmentation
performance on the PASCAL VOC benchmark with these trainable higher order
potentials.
Anurag Arnab Sadeep Jayasumana Shuai Zheng Philip Torr