is224224(with zero-padding) and 404404pixels if the net is applied convolutionally. After
converting the network to a fully convolutional one, the ﬁrst fully connected layer has 4,096 ﬁlters of large 77spatial size and becomes the computational bottleneck in our dense score map
computation.
We have addressed this practical problem by spatially subsampling (by simple decimation) the ﬁrst
FC layer to 44(or33) spatial size. This has reduced the receptive ﬁeld of the network down to
128128(with zero-padding) or 308308(in convolutional mode) and has reduced computation time
for the ﬁrst FC layer by 23times. Using our Caffe-based implementation and a Titan GPU, the
resulting VGG-derived network is very efﬁcient: Given a 306306input image, it produces 3939
4
Published as a conference paper at ICLR 2015
dense raw feature scores at the top of the network at a rate of about 8 frames/sec during testing. The
speed during training is 3 frames/sec. We have also successfully experimented with reducing the
number of channels at the fully connected layers from 4,096 down to 1,024, considerably further
Deep Convolutional Neural Networks (DCNNs) have recently shown state of the
art performance in high level vision tasks, such as image classification and
object detection. This work brings together methods from DCNNs and
probabilistic graphical models for addressing the task of pixel-level
classification (also called "semantic image segmentation"). We show that
responses at the final layer of DCNNs are not sufficiently localized for
accurate object segmentation. This is due to the very invariance properties
that make DCNNs good for high level tasks. We overcome this poor localization
property of deep networks by combining the responses at the final DCNN layer
with a fully connected Conditional Random Field (CRF). Qualitatively, our
"DeepLab" system is able to localize segment boundaries at a level of accuracy
which is beyond previous methods. Quantitatively, our method sets the new
state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching
71.6% IOU accuracy in the test set. We show how these results can be obtained
efficiently: Careful network re-purposing and a novel application of the 'hole'
algorithm from the wavelet community allow dense computation of neural net
responses at 8 frames per second on a modern GPU.
Liang-Chieh Chen George Papandreou Iasonas Kokkinos Kevin Murphy Alan L. Yuille