denseCRF inference requires 3.301 seconds per image on
Input Image
Iteration 0Iteration 50
Figure 6: A ﬁgure illustrating how the probability predictions change as we apply more random walk steps. Note
that the RWN predictions become more reﬁned and better localized around the object boundaries as more random
walk steps are applied.
average on a Pascal SBD dataset. In comparison, a single
iteration of a random walk, which is simply a sparse matrix
multiplication, takes 0.032 seconds on average on the same
Pascal SBD dataset. A DeepLab_v2 post-processed with
denseCRF achieves 81:9%IOU score on this same Pascal
SBD dataset. In comparison, RWN_v2 with a single random walk iteration and with R=40 (radius) achieves 82:2%
IOU, which is both more accurate and more than 100times
more efﬁcient than the denseCRF inference.
5.4. Ablation Experiments
Optimal Number of Random Walk Steps. In Figure 7,
we illustrate how the IOU accuracy changes when we use a
different number of random walk steps. We observe that the
segmentation accuracy keeps increasing as we apply more
random walk steps, and that it reaches its peak performance
when the random walk process converges, which indicates
Most current semantic segmentation methods rely on fully convolutional
networks (FCNs). However, their use of large receptive fields and many pooling
layers cause low spatial resolution inside the deep layers. This leads to
predictions with poor localization around the boundaries. Prior work has
attempted to address this issue by post-processing predictions with CRFs or
MRFs. But such models often fail to capture semantic relationships between
objects, which causes spatially disjoint predictions. To overcome these
problems, recent methods integrated CRFs or MRFs into an FCN framework. The
downside of these new models is that they have much higher complexity than
traditional FCNs, which renders training and testing more challenging.
  In this work we introduce a simple, yet effective Convolutional Random Walk
Network (RWN) that addresses the issues of poor boundary localization and
spatially fragmented predictions with very little increase in model complexity.
Our proposed RWN jointly optimizes the objectives of pixelwise affinity and
semantic segmentation. It combines these two objectives via a novel random walk
layer that enforces consistent spatial grouping in the deep layers of the
network. Our RWN is implemented using standard convolution and matrix
multiplication. This allows an easy integration into existing FCN frameworks
and it enables end-to-end training of the whole network via standard
back-propagation. Our implementation of RWN requires just $131$ additional
parameters compared to the traditional FCNs, and yet it consistently produces
an improvement over the FCNs on semantic segmentation and scene labeling.
Gedas Bertasius Lorenzo Torresani Stella X. Yu Jianbo Shi