improves the performance of all globalization techniques.
Note that all of the inference methods use the same FCN
unary potentials . This suggests that for every method our
boundary-based afﬁnity is more beneﬁcial for semantic segmentation than the color-based afﬁnity.
represents one of the object classes). In the experimental
section, we show that this solution produces better quantitative and qualitative results in comparison to commonly used
globalization techniques.
4. Experimental Results
In this section we present quantitative and qualitative results for semantic segmentation on the SBD [13] dataset,
which contains objects and their per-pixel annotations for
20Pascal VOC classes. We evaluate semantic segmentation results using two evaluation metrics. The ﬁrst metric
measures accuracy based on pixel intersection-over-union
averaged per pixels (PP-IOU) across the 20 classes. According to this metric, the accuracy is computed on a perpixel basis. As a result, the images that contain large object regions are given more importance. However, for certain applications we may need to accurately segment small
objects. Therefore, similar to [4] we also consider the PIIOU metric (pixel intersection-over-union averaged per image across the 20 classes), which gives equal weight to each
of the images.
The state-of-the-art in semantic segmentation is currently represented by
fully convolutional networks (FCNs). However, FCNs use large receptive fields
and many pooling layers, both of which cause blurring and low spatial
resolution in the deep layers. As a result FCNs tend to produce segmentations
that are poorly localized around object boundaries. Prior work has attempted to
address this issue in post-processing steps, for example using a color-based
CRF on top of the FCN predictions. However, these approaches require additional
parameters and low-level features that are difficult to tune and integrate into
the original network architecture. Additionally, most CRFs use color-based
pixel affinities, which are not well suited for semantic segmentation and lead
to spatially disjoint predictions.
  To overcome these problems, we introduce a Boundary Neural Field (BNF), which
is a global energy model integrating FCN predictions with boundary cues. The
boundary information is used to enhance semantic segment coherence and to
improve object localization. Specifically, we first show that the convolutional
filters of semantic FCNs provide good features for boundary detection. We then
employ the predicted boundaries to define pairwise potentials in our energy.
Finally, we show that our energy decomposes semantic segmentation into multiple
binary problems, which can be relaxed for efficient global optimization. We
report extensive experiments demonstrating that minimization of our global
boundary-based energy yields results superior to prior globalization methods,
both quantitatively as well as qualitatively.
Gedas Bertasius Jianbo Shi Lorenzo Torresani