objects ( person ,rider ,bicycle ,motorcycle ), and level 2 (3 to
4 pixels from boundary) for classes with larger objects ( car,
bus,truck ,train ). Following this, the instances are dilated
using a circular structuring element whose radius equals the
maximum erosion at the energy level of the cut. This offsets the boundary erosion from the non-zero threshold. The
connected components are identiﬁed in the resulting image,
directly yielding the proposed instances. The proposals are
further reﬁned by basic hole-ﬁlling. Finally, we remove
small, spurious instances.
5. Experimental Evaluation
In this section, we evaluate our approach on the challenging Cityscapes Instance-Level Semantic Labelling Task [6].
The ofﬁcial benchmark test and validation set results are
found in Tables 1 and 2. We then perform ablation studies with the validation set to examine the performance of
various aspects of our model.
Dataset: The Cityscapes Instance Labelling Task contains 5000 ﬁnely annotated street scene images taken by a
vehicle-mounted camera. Each image has a resolution of
MethodCityscapes Test Set Cityscapes Val Set
AP AP 50% AP 100m AP 50m AP muCov
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun