segmentation tree to smooth the prediction results; more recently, Hariharan et al. (2014a) propose to
concatenate the computed inter-mediate feature maps within the DCNNs for pixel classiﬁcation, and
Dai et al. (2014) propose to pool the inter-mediate feature maps by region proposals. Even though
these works still employ segmentation algorithms that are decoupled from the DCNN classiﬁer’s
results, we believe it is advantageous that segmentation is only used at a later stage, avoiding the
commitment to premature decisions.
More recently, the segmentation-free techniques of (Long et al., 2014; Eigen & Fergus, 2014) directly apply DCNNs to the whole image in a sliding window fashion, replacing the last fully connected layers of a DCNN by convolutional layers. In order to deal with the spatial localization
issues outlined in the beginning of the introduction, Long et al. (2014) upsample and concatenate
the scores from inter-mediate feature maps, while Eigen & Fergus (2014) reﬁne the prediction result
from coarse to ﬁne by propagating the coarse results to another DCNN.
The main difference between our model and other state-of-the-art models is the combination of
Deep Convolutional Neural Networks (DCNNs) have recently shown state of the
art performance in high level vision tasks, such as image classification and
object detection. This work brings together methods from DCNNs and
probabilistic graphical models for addressing the task of pixel-level
classification (also called "semantic image segmentation"). We show that
responses at the final layer of DCNNs are not sufficiently localized for
accurate object segmentation. This is due to the very invariance properties
that make DCNNs good for high level tasks. We overcome this poor localization
property of deep networks by combining the responses at the final DCNN layer
with a fully connected Conditional Random Field (CRF). Qualitatively, our
"DeepLab" system is able to localize segment boundaries at a level of accuracy
which is beyond previous methods. Quantitatively, our method sets the new
state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching
71.6% IOU accuracy in the test set. We show how these results can be obtained
efficiently: Careful network re-purposing and a novel application of the 'hole'
algorithm from the wavelet community allow dense computation of neural net
responses at 8 frames per second on a modern GPU.
Liang-Chieh Chen George Papandreou Iasonas Kokkinos Kevin Murphy Alan L. Yuille