0 5 10 15 20 25 30 35 406668707274767880
Trimap WidthInterior IoU
  
Pairwise only
Superpixels
Detections
Detections and Superpixels
0 5 10 15 20 25 30 35 40505560657075
Trimap WidthBoundary IoU
  
Pairwise only
Superpixels
Detections
Detections and Superpixels
(e) Interior IoU (f) Boundary IoU
Fig. 5: Error analysis on VOC 2012 reduced validation set The IoU is computed for
boundary and interior regions for various trimap widths. An example of the Boundary
and Interior regions for a sample image using a width of 9 pixels is shown in white in
the top row. Black regions are ignored in the IoU calculation.
be due to the fact that superpixels do not always align correctly with the edges of an
object (the “boundary recall” of various superpixel methods are evaluated in [40]).
We can see that the combination of detection and superpixel potentials results in a
substantial improvement in our Interior IoU. This is the primary reason our overall IoU
on the VOC benchmark increases with higher order potentials.
Beneﬁts of end-to-end training Table 4 shows how end-to-end training outperforms
We address the problem of semantic segmentation using deep learning. Most
segmentation systems include a Conditional Random Field (CRF) to produce a
structured output that is consistent with the image's visual features. Recent
deep learning approaches have incorporated CRFs into Convolutional Neural
Networks (CNNs), with some even training the CRF end-to-end with the rest of
the network. However, these approaches have not employed higher order
potentials, which have previously been shown to significantly improve
segmentation performance. In this paper, we demonstrate that two types of
higher order potential, based on object detections and superpixels, can be
included in a CRF embedded within a deep network. We design these higher order
potentials to allow inference with the differentiable mean field algorithm. As
a result, all the parameters of our richer CRF model can be learned end-to-end
with our pixelwise CNN classifier. We achieve state-of-the-art segmentation
performance on the PASCAL VOC benchmark with these trainable higher order
potentials.
Anurag Arnab Sadeep Jayasumana Shuai Zheng Philip Torr