more challenging. We use the ofﬁcial training, validation,
and testing set splits, with 2975, 500, and 1525 images, respectively.
Metric: We report the metrics employed by the
Cityscapes leaderboard. Several variants of the instancelevel average precision score are used. This is calculated
by ﬁnding the precision at various levels of intersectionover-union (IoU) between predicted instances and ground
truth instances, ranging from 50% to 95%. The main score
is the Average Precision (AP). Additionally, there are three
minor scores: AP at 50% overlap, AP of objects closer
than 50m, and AP of objects closer than 100m. Methods
are ranked by the AP score over all classes (mean AP).
Note that AP is a detection score, which does not penalize
overlapping instances or a large number of predictions,
as long as they are ranked in the proper order. Thus,
it places approaches (like ours) which predict a single
instance label (or background) per pixel at a disadvantage,
while favoring detection-based methods. Despite this, we
use these metrics to be consistent with the the evaluation
of the state-of-the-art. For the fbus,truck ,traingclasses,
we order the instances by simply averaging the semantic
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun