[22] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In The IEEE Conference
on Computer Vision and Pattern Recognition (CVPR) , June 2015. 1,
3
[23] D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human
segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. 8th
Int’l Conf. Computer Vision , volume 2, pages 416–423, July 2001. 4
[24] Mohammadreza Mostajabi, Payman Yadollahpour, and Gregory
Shakhnarovich. Feedforward semantic segmentation with zoom-out
features. CoRR , abs/1412.0774, 2014. 2
[25] X. Ren and L. Bo. Discriminatively Trained Sparse Code Gradients
for Contour Detection. In Advances in Neural Information Processing Systems , December 2012. 4
[26] Carsten Rother, Vladimir Kolmogorov, Victor Lempitsky, and Martin
Szummer. Optimizing binary mrfs via extended roof duality. In Proc
Comp. Vision Pattern Recogn. (CVPR) , June 2007. 7, 8
[27] Wei Shen, Xinggang Wang, Yan Wang, Xiang Bai, and Zhijiang
The state-of-the-art in semantic segmentation is currently represented by
fully convolutional networks (FCNs). However, FCNs use large receptive fields
and many pooling layers, both of which cause blurring and low spatial
resolution in the deep layers. As a result FCNs tend to produce segmentations
that are poorly localized around object boundaries. Prior work has attempted to
address this issue in post-processing steps, for example using a color-based
CRF on top of the FCN predictions. However, these approaches require additional
parameters and low-level features that are difficult to tune and integrate into
the original network architecture. Additionally, most CRFs use color-based
pixel affinities, which are not well suited for semantic segmentation and lead
to spatially disjoint predictions.
  To overcome these problems, we introduce a Boundary Neural Field (BNF), which
is a global energy model integrating FCN predictions with boundary cues. The
boundary information is used to enhance semantic segment coherence and to
improve object localization. Specifically, we first show that the convolutional
filters of semantic FCNs provide good features for boundary detection. We then
employ the predicted boundaries to define pairwise potentials in our energy.
Finally, we show that our energy decomposes semantic segmentation into multiple
binary problems, which can be relaxed for efficient global optimization. We
report extensive experiments demonstrating that minimization of our global
boundary-based energy yields results superior to prior globalization methods,
both quantitatively as well as qualitatively.
Gedas Bertasius Jianbo Shi Lorenzo Torresani