to the regions proposals delivered by (Carreira & Sminchisescu, 2012). Understanding the perils
of committing to a single segmentation, the authors of Cogswell et al. (2014) build on (Yadollahpour et al., 2013) to explore a diverse set of CRF-based segmentation proposals, computed also by
(Carreira & Sminchisescu, 2012). These segmentation proposals are then re-ranked according to a
DCNN trained in particular for this reranking task. Even though this approach explicitly tries to
handle the temperamental nature of a front-end segmentation algorithm, there is still no explicit ex2
Published as a conference paper at ICLR 2015
ploitation of the DCNN scores in the CRF-based segmentation algorithm: the DCNN is only applied
post-hoc, while it would make sense to directly try to use its results during segmentation.
Moving towards works that lie closer to our approach, several other researchers have considered
the use of convolutionally computed DCNN features for dense image labeling. Among the ﬁrst
have been Farabet et al. (2013) who apply DCNNs at multiple image resolutions and then employ a
segmentation tree to smooth the prediction results; more recently, Hariharan et al. (2014a) propose to
Deep Convolutional Neural Networks (DCNNs) have recently shown state of the
art performance in high level vision tasks, such as image classification and
object detection. This work brings together methods from DCNNs and
probabilistic graphical models for addressing the task of pixel-level
classification (also called "semantic image segmentation"). We show that
responses at the final layer of DCNNs are not sufficiently localized for
accurate object segmentation. This is due to the very invariance properties
that make DCNNs good for high level tasks. We overcome this poor localization
property of deep networks by combining the responses at the final DCNN layer
with a fully connected Conditional Random Field (CRF). Qualitatively, our
"DeepLab" system is able to localize segment boundaries at a level of accuracy
which is beyond previous methods. Quantitatively, our method sets the new
state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching
71.6% IOU accuracy in the test set. We show how these results can be obtained
efficiently: Careful network re-purposing and a novel application of the 'hole'
algorithm from the wavelet community allow dense computation of neural net
responses at 8 frames per second on a modern GPU.
Liang-Chieh Chen George Papandreou Iasonas Kokkinos Kevin Murphy Alan L. Yuille