PASCAL VOC 2012 Dataset The improvement obtained by each higher order potential
was evaluated on the same reduced validation set [3] used by our baseline [10]. As Table
1 shows, each new higher order potential improves the mean IoU over the baseline. We
only report test set results for our best method since the VOC guidelines discourage
the use of the test set for ablation studies. On the test set (Table 2), we outperform our
baseline by 3.2% which equates to a 12.6% reduction in the error rate. This sets a new
state-of-the-art on the VOC dataset. Qualitative results highlighting success and failure
cases of our algorithm, as well as more detailed results, are shown in our supplementary
material.
PASCAL Context Table 3 shows our state-of-the-art results on the recently released
PASCAL Context dataset [50]. We trained on the provided training set of 4998 images,
and evaluated on the validation set of 5105 images. This dataset augments VOC with
annotations for all objects in the scene. As a result, there are 59 classes as opposed to
the 20 in the VOC dataset. Many of these new labels are “stuff” classes such as “grass”
and “sky”. Our object detectors are therefore only trained for 20 of the 59 labels in this
We address the problem of semantic segmentation using deep learning. Most
segmentation systems include a Conditional Random Field (CRF) to produce a
structured output that is consistent with the image's visual features. Recent
deep learning approaches have incorporated CRFs into Convolutional Neural
Networks (CNNs), with some even training the CRF end-to-end with the rest of
the network. However, these approaches have not employed higher order
potentials, which have previously been shown to significantly improve
segmentation performance. In this paper, we demonstrate that two types of
higher order potential, based on object detections and superpixels, can be
included in a CRF embedded within a deep network. We design these higher order
potentials to allow inference with the differentiable mean field algorithm. As
a result, all the parameters of our richer CRF model can be learned end-to-end
with our pixelwise CNN classifier. We achieve state-of-the-art segmentation
performance on the PASCAL VOC benchmark with these trainable higher order
potentials.
Anurag Arnab Sadeep Jayasumana Shuai Zheng Philip Torr