into an efﬁcient and effective dense feature extractor for our dense semantic image segmentation
system.
3.1 E FFICIENT DENSE SLIDING WINDOW FEATURE EXTRACTION WITH THE HOLE
ALGORITHM
Dense spatial score evaluation is instrumental in the success of our dense CNN feature extractor. As
a ﬁrst step to implement this, we convert the fully-connected layers of VGG-16 into convolutional
ones and run the network in a convolutional fashion on the image at its original resolution. However
this is not enough as it yields very sparsely computed detection scores (with a stride of 32 pixels). To
compute scores more densely at our target stride of 8 pixels, we develop a variation of the method
previously employed by Giusti et al. (2013); Sermanet et al. (2013). We skip subsampling after
the last two max-pooling layers in the network of Simonyan & Zisserman (2014) and modify the
convolutional ﬁlters in the layers that follow them by introducing zeros to increase their length ( 2in
3
Published as a conference paper at ICLR 2015
Input stride
Output stride
Figure 1: Illustration of the hole algorithm in 1-D, when kernel size = 3 ,input stride = 2 , and
Deep Convolutional Neural Networks (DCNNs) have recently shown state of the
art performance in high level vision tasks, such as image classification and
object detection. This work brings together methods from DCNNs and
probabilistic graphical models for addressing the task of pixel-level
classification (also called "semantic image segmentation"). We show that
responses at the final layer of DCNNs are not sufficiently localized for
accurate object segmentation. This is due to the very invariance properties
that make DCNNs good for high level tasks. We overcome this poor localization
property of deep networks by combining the responses at the final DCNN layer
with a fully connected Conditional Random Field (CRF). Qualitatively, our
"DeepLab" system is able to localize segment boundaries at a level of accuracy
which is beyond previous methods. Quantitatively, our method sets the new
state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching
71.6% IOU accuracy in the test set. We show how these results can be obtained
efficiently: Careful network re-purposing and a novel application of the 'hole'
algorithm from the wavelet community allow dense computation of neural net
responses at 8 frames per second on a modern GPU.
Liang-Chieh Chen George Papandreou Iasonas Kokkinos Kevin Murphy Alan L. Yuille