resolution. We utilize a modiﬁed version of the ﬁrst 13
layers of VGG, where the third and fourth pooling layers are changed to average pooling while the ﬁfth pooling
layer is removed. To preserve spatial resolution, we exploit a high-capacity, multi-scale information aggregation
scheme inspired by popular methods in semantic segmenta-tion [9, 30, 16]. In particular, the outputs of conv3 ,conv4 ,
andconv5 individually undergo a 55convolution followed
by two 11convolutions. After this, the outputs of the
second and third paths are upsampled to the resolution of
the ﬁrst path. The resulting feature volumes from the three
paths are concatenated. This undergoes an additional set
of three 11convolutions, before being upsampled to the
input resolution.
4.2. Watershed Transform Network (WTN)
The second half of the overall network takes as input the
2-channel unit vector map, and produces a discretized modiﬁed watershed transform map with K= 16 possible energy
values. In this case, bin 0 corresponds to background or regions within 2 pixels of an instance boundary and is referred
to as having an energy level of 0. Meanwhile, higher numbered bins with higher energy levels correspond to regions
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun