from the detection bounding boxes. These choices were made after conducting preliminary experiments with alternate detection and foreground segmentation algorithms.
Four levels of superpixel oversegmentations were used, with increasing superpixel
size to deﬁne the cliques used in this potential. Four levels were used since performance
on the VOC validation set stopped increasing after this number. We used the superpixel
method of [39] as it was shown to adhere to object boundaries the best [40], but our
method generalises to any oversegmentation algorithm.
We trained the full network end-to-end, optimising the weights of the CNN classiﬁer
(FCN-8s) and CRF parameters jointly. We initialised our network using the publicly
available weights of [10], and trained with a learning rate of 10−10and momentum of
0.99. The learning rate is low because the loss was not normalised by the number of
pixels in the training image. This is to have a larger loss for images with more pixels.
When training our CRF, we only used VOC 2012 data [4] as it has the most accurate
labelling, particularly around boundaries.
12 Arnab et al.
PASCAL VOC 2012 Dataset The improvement obtained by each higher order potential
was evaluated on the same reduced validation set [3] used by our baseline [10]. As Table
We address the problem of semantic segmentation using deep learning. Most
segmentation systems include a Conditional Random Field (CRF) to produce a
structured output that is consistent with the image's visual features. Recent
deep learning approaches have incorporated CRFs into Convolutional Neural
Networks (CNNs), with some even training the CRF end-to-end with the rest of
the network. However, these approaches have not employed higher order
potentials, which have previously been shown to significantly improve
segmentation performance. In this paper, we demonstrate that two types of
higher order potential, based on object detections and superpixels, can be
included in a CRF embedded within a deep network. We design these higher order
potentials to allow inference with the differentiable mean field algorithm. As
a result, all the parameters of our richer CRF model can be learned end-to-end
with our pixelwise CNN classifier. We achieve state-of-the-art segmentation
performance on the PASCAL VOC benchmark with these trainable higher order
potentials.
Anurag Arnab Sadeep Jayasumana Shuai Zheng Philip Torr