to as having an energy level of 0. Meanwhile, higher numbered bins with higher energy levels correspond to regions
in the interior of object instances. The bins are chosen to
maximize the binning resolution of energy levels near zero
(to facilitate accurate cutting), while achieving an approximate balance between the total numbers of pixels within
each class.
The WTN is a fairly generic CNN, with an emphasis on
reasoning with high spatial accuracy and resolution. The architecture is shown in the right half of Fig. 3. In particular,
the network consists of two 55convolutional ﬁlter blocks
each followed by a 22average pooling, before undergoing a set of 11convolutions and upsampling to the input
image resolution.
4.3. Network Training
We ﬁrst pre-train our DN and WTN networks. This process is followed by end-to-end training of the whole network.
Direction Network pre-training: We pre-train the network using the mean squared error in the angular domain as
loss:
ldirection =X
p2P objwpkcos1< ~ up;GT; ~ up;pred>k2
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun