while all dividing ridges are at the same height in the energy domain. As a consequence, the components can be
extracted by a cut at a single energy level without leading
to over-segmentation. Our approach has several key advantages: it can be easily trained end-to-end, and produces very
fast and accurate estimates. Our method does not rely on iterative strategies such as RNNs, thus has a constant runtime
regardless of the number of object instances.
We demonstrate the effectiveness of our approach in the
challenging Cityscapes Instance Segmentation benchmark
[6], and show that we more than double the performance
of the current state-of-the-art. In the following sections, we
ﬁrst review related work. We then present the details behind
our intuition and model design, followed by an analysis of
our model’s performance. Finally, we explore the impact of
various parts of our model in ablation studies.
1arXiv:1611.08303v2  [cs.CV]  4 May 2017
(a) Input Image
 (b) Semantic Segmentation [34]
 (c) Our Instance Segmentation
 (d) GT Instance Segmentation
Figure 1: Sample prediction: the input image is gated by sem. segmentation from [34] and passed through our model.
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun