in the augmented ‘train’ set). The best performance is achieved by exploiting both multi-scale
features and large ﬁeld-of-view. (b) Performance of our proposed models (with training in the
augmented ‘trainval’ set) compared to other state-of-art methods on the PASCAL VOC 2012 ‘test’
set.
5 E XPERIMENTAL EVALUATION
Dataset We test our DeepLab model on the PASCAL VOC 2012 segmentation benchmark (Everingham et al., 2014), consisting of 20 foreground object classes and one background class. The
original dataset contains 1;464,1;449, and 1;456images for training, validation, and testing, respectively. The dataset is augmented by the extra annotations provided by Hariharan et al. (2011),
resulting in 10;582training images. The performance is measured in terms of pixel intersectionover-union (IOU) averaged across the 21 classes.
Training We adopt the simplest form of piecewise training, decoupling the DCNN and CRF training stages, assuming the unary terms provided by the DCNN are ﬁxed during CRF training.
For DCNN training we employ the VGG-16 network which has been pre-trained on ImageNet. We
Deep Convolutional Neural Networks (DCNNs) have recently shown state of the
art performance in high level vision tasks, such as image classification and
object detection. This work brings together methods from DCNNs and
probabilistic graphical models for addressing the task of pixel-level
classification (also called "semantic image segmentation"). We show that
responses at the final layer of DCNNs are not sufficiently localized for
accurate object segmentation. This is due to the very invariance properties
that make DCNNs good for high level tasks. We overcome this poor localization
property of deep networks by combining the responses at the final DCNN layer
with a fully connected Conditional Random Field (CRF). Qualitatively, our
"DeepLab" system is able to localize segment boundaries at a level of accuracy
which is beyond previous methods. Quantitatively, our method sets the new
state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching
71.6% IOU accuracy in the test set. We show how these results can be obtained
efficiently: Careful network re-purposing and a novel application of the 'hole'
algorithm from the wavelet community allow dense computation of neural net
responses at 8 frames per second on a modern GPU.
Liang-Chieh Chen George Papandreou Iasonas Kokkinos Kevin Murphy Alan L. Yuille