Zdexp/parenleftbig
−Et(Yd=b)/parenrightbig
,∀d,b
⊿Normalising
end for
For the explicit forms of the UnaryUpdate andPairwiseUpdate above, and their
differentials, we refer the reader to [10] and discuss the terms DetectionUpdate and
SuperpixelUpdate in detail below.
Let us assume that only one object detection of the form (ld,sd,Fd)is available for
the image under consideration. When multiple detections are present, simply a summation of the updates and differentials discussed below apply. Therefore, no generality
18 Arnab et al.
is lost with this assumption. Similarly, we can assume that only one superpixel clique
{Xs}is present, without a loss of generality.
Assuming that pixel iin Algorithm 1 belongs to Fd, Eq. (6) in the main paper
described the exact form of DetectionUpdate . Similarly, assuming that pixel ibelongs
to{Xs}Eq. (8) described the form of SuperpixelUpdate .
LetLdenote the value of the loss function calculated at the output of the deep
network. This could be the softmax loss or any other appropriate loss function. During
backpropagation, we get the error signal∂L
We address the problem of semantic segmentation using deep learning. Most
segmentation systems include a Conditional Random Field (CRF) to produce a
structured output that is consistent with the image's visual features. Recent
deep learning approaches have incorporated CRFs into Convolutional Neural
Networks (CNNs), with some even training the CRF end-to-end with the rest of
the network. However, these approaches have not employed higher order
potentials, which have previously been shown to significantly improve
segmentation performance. In this paper, we demonstrate that two types of
higher order potential, based on object detections and superpixels, can be
included in a CRF embedded within a deep network. We design these higher order
potentials to allow inference with the differentiable mean field algorithm. As
a result, all the parameters of our richer CRF model can be learned end-to-end
with our pixelwise CNN classifier. We achieve state-of-the-art segmentation
performance on the PASCAL VOC benchmark with these trainable higher order
potentials.
Anurag Arnab Sadeep Jayasumana Shuai Zheng Philip Torr