however their increased invariance and large receptive ﬁelds make the problem of inferring position
from the scores at their top output levels more challenging.
Recent work has pursued two directions to address this localization challenge. The ﬁrst approach is
to harness information from multiple layers in the convolutional network in order to better estimate
the object boundaries (Long et al., 2014; Eigen & Fergus, 2014). The second approach is to employ
a super-pixel representation, essentially delegating the localization task to a low-level segmentation
method. This route is followed by the very successful recent method of Mostajabi et al. (2014).
In Section 4.2, we pursue a novel alternative direction based on coupling the recognition capacity
of DCNNs and the ﬁne-grained localization accuracy of fully connected CRFs and show that it is
remarkably successful in addressing the localization challenge, producing accurate semantic segmentation results and recovering object boundaries at a level of detail that is well beyond the reach
of existing methods.
4.2 F ULLY -CONNECTED CONDITIONAL RANDOM FIELDS FOR ACCURATE LOCALIZATION
Image/G.T. DCNN output CRF Iteration 1 CRF Iteration 2 CRF Iteration 10
Deep Convolutional Neural Networks (DCNNs) have recently shown state of the
art performance in high level vision tasks, such as image classification and
object detection. This work brings together methods from DCNNs and
probabilistic graphical models for addressing the task of pixel-level
classification (also called "semantic image segmentation"). We show that
responses at the final layer of DCNNs are not sufficiently localized for
accurate object segmentation. This is due to the very invariance properties
that make DCNNs good for high level tasks. We overcome this poor localization
property of deep networks by combining the responses at the final DCNN layer
with a fully connected Conditional Random Field (CRF). Qualitatively, our
"DeepLab" system is able to localize segment boundaries at a level of accuracy
which is beyond previous methods. Quantitatively, our method sets the new
state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching
71.6% IOU accuracy in the test set. We show how these results can be obtained
efficiently: Careful network re-purposing and a novel application of the 'hole'
algorithm from the wavelet community allow dense computation of neural net
responses at 8 frames per second on a modern GPU.
Liang-Chieh Chen George Papandreou Iasonas Kokkinos Kevin Murphy Alan L. Yuille