images using the publicly available implementation of (Kr ¨ahenb ¨uhl & Koltun, 2011).
4.3 M ULTI -SCALE PREDICTION
Following the promising recent results of (Hariharan et al., 2014a; Long et al., 2014) we have also
explored a multi-scale prediction method to increase the boundary localization accuracy. Specifically, we attach to the input image and the output of each of the ﬁrst four max pooling layers a
two-layer MLP (ﬁrst layer: 128 3x3 convolutional ﬁlters, second layer: 128 1x1 convolutional ﬁlters) whose feature map is concatenated to the main network’s last layer feature map. The aggregate
feature map fed into the softmax layer is thus enhanced by 5 * 128 = 640 channels. We only adjust
the newly added weights, keeping the other network parameters to the values learned by the method
of Section 3. As discussed in the experimental section, introducing these extra direct connections
from ﬁne-resolution layers improves localization performance, yet the effect is not as dramatic as
the one obtained with the fully-connected CRF.
6
Published as a conference paper at ICLR 2015
Method mean IOU (%)
DeepLab 59.80
DeepLab-CRF 63.74
Deep Convolutional Neural Networks (DCNNs) have recently shown state of the
art performance in high level vision tasks, such as image classification and
object detection. This work brings together methods from DCNNs and
probabilistic graphical models for addressing the task of pixel-level
classification (also called "semantic image segmentation"). We show that
responses at the final layer of DCNNs are not sufficiently localized for
accurate object segmentation. This is due to the very invariance properties
that make DCNNs good for high level tasks. We overcome this poor localization
property of deep networks by combining the responses at the final DCNN layer
with a fully connected Conditional Random Field (CRF). Qualitatively, our
"DeepLab" system is able to localize segment boundaries at a level of accuracy
which is beyond previous methods. Quantitatively, our method sets the new
state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching
71.6% IOU accuracy in the test set. We show how these results can be obtained
efficiently: Careful network re-purposing and a novel application of the 'hole'
algorithm from the wavelet community allow dense computation of neural net
responses at 8 frames per second on a modern GPU.
Liang-Chieh Chen George Papandreou Iasonas Kokkinos Kevin Murphy Alan L. Yuille