globalization approaches, our goal is to minimize the energy
corresponding to the sum of unary and pairwise potentials.
However, the key difference in our approach comes from the
relaxation of some of the constraints. Speciﬁcally, instead
of modeling our problem as a joint multi-label distribution,
we propose to decompose it into multiple binary problems,
which can be solved concurrently. This decomposition can
be viewed as assigning pixels to foreground and background
labels for each of the different object classes. Additionally,
we relax the integrality constraint. Both of these relaxations
make our problem more manageable and allow us to formulate a global energy function that is differentiable, and has
a closed form solution.
In [35], the authors introduce the idea of learning
with global and local consistency in the context of semisupervised problems. Inspired by this work, we incorporate
some of these ideas in the context of semantic segmentation. Before deﬁning our proposed global energy function,
we introduce some relevant notation.
For the purpose of illustration, suppose that we only have
two classes: foreground and background. Then we can denote an optimal continuous solution to such a segmentation
problem with variable z. To denote similarity between pixelsiandjwe usewij. Then,diindicates the degree of a
The state-of-the-art in semantic segmentation is currently represented by
fully convolutional networks (FCNs). However, FCNs use large receptive fields
and many pooling layers, both of which cause blurring and low spatial
resolution in the deep layers. As a result FCNs tend to produce segmentations
that are poorly localized around object boundaries. Prior work has attempted to
address this issue in post-processing steps, for example using a color-based
CRF on top of the FCN predictions. However, these approaches require additional
parameters and low-level features that are difficult to tune and integrate into
the original network architecture. Additionally, most CRFs use color-based
pixel affinities, which are not well suited for semantic segmentation and lead
to spatially disjoint predictions.
  To overcome these problems, we introduce a Boundary Neural Field (BNF), which
is a global energy model integrating FCN predictions with boundary cues. The
boundary information is used to enhance semantic segment coherence and to
improve object localization. Specifically, we first show that the convolutional
filters of semantic FCNs provide good features for boundary detection. We then
employ the predicted boundaries to define pairwise potentials in our energy.
Finally, we show that our energy decomposes semantic segmentation into multiple
binary problems, which can be relaxed for efficient global optimization. We
report extensive experiments demonstrating that minimization of our global
boundary-based energy yields results superior to prior globalization methods,
both quantitatively as well as qualitatively.
Gedas Bertasius Jianbo Shi Lorenzo Torresani