d=ld] ifyd= 0,
wDetsd
nd[x(i)
d/ne}ationslash=ld] ifyd= 1.(3)
We make use of this simpliﬁcation in Section 5 when deriving the mean ﬁeld updates
associated with this potential.
For the latent Yvariables, in addition to the joint potentials with Xvariables,
described in Eq. (2)and(3), we also include unary potentials, which are initialised
from the score sdof the object detection. The underlying idea is that if the object
detector detects an object with high conﬁdence, the CRF in turn starts with a high initial
conﬁdence about the validity of that detection. This conﬁdence can, of course, change
during the CRF inference depending on other information ( e.g.segmentation unary
potentials) available to the CRF.
Examples of input images with multiple detections and GrabCut foreground masks
are shown in Figure 3. Note how false detections are ignored and erroneous parts of the
foreground mask are also largely ignored.
4.2 Superpixel Based Potentials
The next type of higher order potential we use is based on the idea that superpixels
We address the problem of semantic segmentation using deep learning. Most
segmentation systems include a Conditional Random Field (CRF) to produce a
structured output that is consistent with the image's visual features. Recent
deep learning approaches have incorporated CRFs into Convolutional Neural
Networks (CNNs), with some even training the CRF end-to-end with the rest of
the network. However, these approaches have not employed higher order
potentials, which have previously been shown to significantly improve
segmentation performance. In this paper, we demonstrate that two types of
higher order potential, based on object detections and superpixels, can be
included in a CRF embedded within a deep network. We design these higher order
potentials to allow inference with the differentiable mean field algorithm. As
a result, all the parameters of our richer CRF model can be learned end-to-end
with our pixelwise CNN classifier. We achieve state-of-the-art segmentation
performance on the PASCAL VOC benchmark with these trainable higher order
potentials.
Anurag Arnab Sadeep Jayasumana Shuai Zheng Philip Torr