We ﬁnetune the model weights of the Imagenet-pretrained VGG-16 network to adapt it to the image
classiﬁcation task in a straightforward fashion, following the procedure of Long et al. (2014). We
replace the 1000-way Imagenet classiﬁer in the last layer of VGG-16 with a 21-way one. Our
loss function is the sum of cross-entropy terms for each spatial position in the CNN output map
(subsampled by 8 compared to the original image). All positions and labels are equally weighted in
the overall loss function. Our targets are the ground truth labels (subsampled by 8). We optimize the
objective function with respect to the weights at all network layers by the standard SGD procedure
of Krizhevsky et al. (2013).
During testing, we need class score maps at the original image resolution. As illustrated in Figure 2
and further elaborated in Section 4.1, the class score maps (corresponding to log-probabilities) are
quite smooth, which allows us to use simple bilinear interpolation to increase their resolution by a
factor of 8 at a negligible computational cost. Note that the method of Long et al. (2014) does not
use the hole algorithm and produces very coarse scores (subsampled by a factor of 32) at the CNN
Deep Convolutional Neural Networks (DCNNs) have recently shown state of the
art performance in high level vision tasks, such as image classification and
object detection. This work brings together methods from DCNNs and
probabilistic graphical models for addressing the task of pixel-level
classification (also called "semantic image segmentation"). We show that
responses at the final layer of DCNNs are not sufficiently localized for
accurate object segmentation. This is due to the very invariance properties
that make DCNNs good for high level tasks. We overcome this poor localization
property of deep networks by combining the responses at the final DCNN layer
with a fully connected Conditional Random Field (CRF). Qualitatively, our
"DeepLab" system is able to localize segment boundaries at a level of accuracy
which is beyond previous methods. Quantitatively, our method sets the new
state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching
71.6% IOU accuracy in the test set. We show how these results can be obtained
efficiently: Careful network re-purposing and a novel application of the 'hole'
algorithm from the wavelet community allow dense computation of neural net
responses at 8 frames per second on a modern GPU.
Liang-Chieh Chen George Papandreou Iasonas Kokkinos Kevin Murphy Alan L. Yuille