and “sky”. Our object detectors are therefore only trained for 20 of the 59 labels in this
dataset. Nevertheless, we improve by 0.8% over the previous state-of-the-art [35] and
2% over our baseline [10].
6.2 Ablation Studies
We perform additional experiments to determine the errors made by our system, show
the beneﬁts of end-to-end training and compare our detection potentials to a simpler
baseline. Unless otherwise stated, these experiments are performed on the VOC 2012
reduced validation set.
Error Analysis To analyse the improvements made by our higher order potentials, we
separately evaluate the performance on the “boundary” and “interior” regions in a similar
manner to [35]. As shown in Fig. 5 c) and d), we consider a narrow band (trimap [16])
around the “void” labels annotated in the VOC 2012 reduced validation set. The mean
IoU of pixels lying within this band is termed the “Boundary IoU” whilst the “Interior
IoU” is evaluated outside this region.
Fig. 5 shows our results as the trimap width is varied. Adding the detection potentials
We address the problem of semantic segmentation using deep learning. Most
segmentation systems include a Conditional Random Field (CRF) to produce a
structured output that is consistent with the image's visual features. Recent
deep learning approaches have incorporated CRFs into Convolutional Neural
Networks (CNNs), with some even training the CRF end-to-end with the rest of
the network. However, these approaches have not employed higher order
potentials, which have previously been shown to significantly improve
segmentation performance. In this paper, we demonstrate that two types of
higher order potential, based on object detections and superpixels, can be
included in a CRF embedded within a deep network. We design these higher order
potentials to allow inference with the differentiable mean field algorithm. As
a result, all the parameters of our richer CRF model can be learned end-to-end
with our pixelwise CNN classifier. We achieve state-of-the-art segmentation
performance on the PASCAL VOC benchmark with these trainable higher order
potentials.
Anurag Arnab Sadeep Jayasumana Shuai Zheng Philip Torr