HFL [4], our method does not require a pre-processing step
to select candidate contour points, as we predict boundaries
on all pixels in the image. We demonstrate that our approach allows us to achieve state-of-the-art boundary detection results according to both F-score and Average Precision metrics. Additionally, due to the semantic nature of
our boundaries, we can successfully use them as pairwise
potentials for semantic segmentation in order to improve
object localization and recover ﬁne structural details, typically lost by pure FCN-based approaches.
Semantic Segmentation. We can group most semantic segmentation methods into three broad categories. The
ﬁrst category can be described as “two-stage” approaches,
where an image is ﬁrst segmented and then each segment
is classiﬁed as belonging to a certain object class. Some
of the most notable methods that belong to this genre include [24, 6, 12, 14].
The primary weakness of the above methods is that they
are unable to recover from errors made by the segmentation
algorithm. Several recent papers [15, 10] address this issue
by proposing to use deep per-pixel CNN features and then
classify each pixel as belonging to a certain class. While
these approaches partially address the incorrect segmentation problem, they perform predictions independently on
The state-of-the-art in semantic segmentation is currently represented by
fully convolutional networks (FCNs). However, FCNs use large receptive fields
and many pooling layers, both of which cause blurring and low spatial
resolution in the deep layers. As a result FCNs tend to produce segmentations
that are poorly localized around object boundaries. Prior work has attempted to
address this issue in post-processing steps, for example using a color-based
CRF on top of the FCN predictions. However, these approaches require additional
parameters and low-level features that are difficult to tune and integrate into
the original network architecture. Additionally, most CRFs use color-based
pixel affinities, which are not well suited for semantic segmentation and lead
to spatially disjoint predictions.
  To overcome these problems, we introduce a Boundary Neural Field (BNF), which
is a global energy model integrating FCN predictions with boundary cues. The
boundary information is used to enhance semantic segment coherence and to
improve object localization. Specifically, we first show that the convolutional
filters of semantic FCNs provide good features for boundary detection. We then
employ the predicted boundaries to define pairwise potentials in our energy.
Finally, we show that our energy decomposes semantic segmentation into multiple
binary problems, which can be relaxed for efficient global optimization. We
report extensive experiments demonstrating that minimization of our global
boundary-based energy yields results superior to prior globalization methods,
both quantitatively as well as qualitatively.
Gedas Bertasius Jianbo Shi Lorenzo Torresani