between two objects will have target unit vectors pointing
in opposite directions. Here, associating a pixel with the
wrong object instance incurs the maximum possible angular
error. This is true regardless of the shape of the objects, even
when they are highly concave or elongated. This forces the
network to learn very accurate boundary localization at the
pixel level.
LetDgt(p)be the ground truth distance transform from
a point to the boundary of the instance it belongs to. We
deﬁne our ground truth targets as the normalized gradient
~ upof this distance transform. More formally,
~ up;gt=rDgt(p)
jrDgt(p)j
Thus, the DN produces a two channel output at the input
image’s resolution representing the directional unit vector.
It is important to note that the normalization layer at the output of the DN restricts the sum of each channel’s squared
output to be 1. This greatly reduces the difﬁculty of using the output of non-linear activation functions to model
Figure 3: Complete network architecture. The network takes the original RGB image gated by semantic segmentation and
concatenated with the semantic segmentation as input, and produces the deep watershed transform energy map as the output.
(a) Input Image
 (b) GT angle of ~ up
 (c) GT Watershed Energy
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun