(a) Input Image
 (b) GT angle of ~ up
 (c) GT Watershed Energy
 (d) GT Instances
(e) Sem. Segmentation of [34]
 (f) Pred. angle of ~ up
 (g) Pred. Watershed Transform
 (h) Pred. Instances
Figure 4: Our network takes the RGB image (a) and the semantic segmentation (e) as input, and predicts a unit vector at each
foreground pixel pointing directly away from the nearest boundary (f). Based on this, we then predict a modiﬁed watershed
transform energy (g), upon which we perform cut at a ﬁxed threshold to yield the ﬁnal predictions (h).
a scalar target. Additionally, phrasing the training target as
a unit vector instead of an angle bypasses the problem of
the equivalent angles 0and 360having a large numerical
difference.
The feature extraction portion of the DN’s architecture
is inspired by VGG16 [27]. However, there are several important modiﬁcations. Direction prediction is a very precise task, as the output can vary greatly between neighboring pixels. Thus, it is critical to avoid losing spatial
resolution. We utilize a modiﬁed version of the ﬁrst 13
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun