results suggest that RWN produces spatially smoother segmentations than DeepLab-CRF.
5.2. Scene Labeling
We also tested our RWN on the task of scene labeling
using two popular datasets: Stanford Background [9] and
Sift Flow [18]. Stanford Background is a relatively small
dataset for scene labeling. It contains 715images, which
we randomly split into 600training images and 115testing images. In contrast, the Sift Flow dataset contains 2489
training examples and 201testing images. For all of our
experiments, we use the DeepLab-largeFOV [6] architecture since it is smaller and more efﬁcient to train and test.
To evaluate scene labeling results, we use the overall IOU
evaluation metric which is a commonly used metric for this
task. In Table 5, we present our scene labeling results on
both of these datasets. Our results indicate that our RWN
method outperforms the DeepLab baseline by 2:57%, and
2:54% on these two datasets, respectively.
5.3. Runtime Comparisons
We also include the runtime comparison of our RWN approach versus the denseCRF inference. We note that using a single core of a 2:7GHz Intel Core i7 processor, the
denseCRF inference requires 3.301 seconds per image on
Input Image
Iteration 0Iteration 50
Most current semantic segmentation methods rely on fully convolutional
networks (FCNs). However, their use of large receptive fields and many pooling
layers cause low spatial resolution inside the deep layers. This leads to
predictions with poor localization around the boundaries. Prior work has
attempted to address this issue by post-processing predictions with CRFs or
MRFs. But such models often fail to capture semantic relationships between
objects, which causes spatially disjoint predictions. To overcome these
problems, recent methods integrated CRFs or MRFs into an FCN framework. The
downside of these new models is that they have much higher complexity than
traditional FCNs, which renders training and testing more challenging.
  In this work we introduce a simple, yet effective Convolutional Random Walk
Network (RWN) that addresses the issues of poor boundary localization and
spatially fragmented predictions with very little increase in model complexity.
Our proposed RWN jointly optimizes the objectives of pixelwise affinity and
semantic segmentation. It combines these two objectives via a novel random walk
layer that enforces consistent spatial grouping in the deep layers of the
network. Our RWN is implemented using standard convolution and matrix
multiplication. This allows an easy integration into existing FCN frameworks
and it enables end-to-end training of the whole network via standard
back-propagation. Our implementation of RWN requires just $131$ additional
parameters compared to the traditional FCNs, and yet it consistently produces
an improvement over the FCNs on semantic segmentation and scene labeling.
Gedas Bertasius Lorenzo Torresani Stella X. Yu Jianbo Shi