work produce score maps and semantic label predictions which are qualitatively different. As illustrated in Figure 2, the score maps are typically quite smooth and produce homogeneous classiﬁcation
results. In this regime, using short-range CRFs can be detrimental, as our goal should be to recover
detailed local structure rather than further smooth it. Using contrast-sensitive potentials (Rother
5
Published as a conference paper at ICLR 2015
Deep 
Convolutional 
Neural 
Network
InputAeroplane
Coarse Score map
Bi-linear Interpolation Fully Connected CRF Final Output
Figure 3: Model Illustration. The coarse score map from Deep Convolutional Neural Network (with
fully convolutional layers) is upsampled by bi-linear interpolation. A fully connected CRF is applied
to reﬁne the segmentation result. Best viewed in color.
et al., 2004) in conjunction to local-range CRFs can potentially improve localization but still miss
thin-structures and typically requires solving an expensive discrete optimization problem.
To overcome these limitations of short-range CRFs, we integrate into our system the fully connected
CRF model of Kr ¨ahenb ¨uhl & Koltun (2011). The model employs the energy function
E(x) =X
ii(xi) +X
Deep Convolutional Neural Networks (DCNNs) have recently shown state of the
art performance in high level vision tasks, such as image classification and
object detection. This work brings together methods from DCNNs and
probabilistic graphical models for addressing the task of pixel-level
classification (also called "semantic image segmentation"). We show that
responses at the final layer of DCNNs are not sufficiently localized for
accurate object segmentation. This is due to the very invariance properties
that make DCNNs good for high level tasks. We overcome this poor localization
property of deep networks by combining the responses at the final DCNN layer
with a fully connected Conditional Random Field (CRF). Qualitatively, our
"DeepLab" system is able to localize segment boundaries at a level of accuracy
which is beyond previous methods. Quantitatively, our method sets the new
state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching
71.6% IOU accuracy in the test set. We show how these results can be obtained
efficiently: Careful network re-purposing and a novel application of the 'hole'
algorithm from the wavelet community allow dense computation of neural net
responses at 8 frames per second on a modern GPU.
Liang-Chieh Chen George Papandreou Iasonas Kokkinos Kevin Murphy Alan L. Yuille