of the state-of-the-art. For the fbus,truck ,traingclasses,
we order the instances by simply averaging the semantic
segmentation’s output conﬁdence within each instance to
somewhat counteract the errors by semantic segmentation.Category Total
PixelsTotal
InstancesAverage
SizeInstances
/ Image
Person 6.6e+7 17900 3691 6.0
Rider 7.4e+6 1754 4244 0.6
Car 3.8e+8 26929 14233 9.1
Truck 1.5e+7 482 30648 0.2
Bus 1.3e+7 379 34275 0.1
Train 3.2e+6 168 19143 0.1
Motorcycle 5.4e+6 735 7298 0.2
Bicycle 1.6e+7 3655 4332 1.2
Table 3: Statistics of the training set of the cityscapes instance level segmentation. Average size is in pixels.
Sem. Seg. Src. Sem. Seg. IoU Inst. Seg. AP
LRR [9] 71.8% 20.2%
PSPNet [34] 80.2% 21.2%
Table 4: Comparison of instance segmentation performance
on the validation set with various semantic segmentation
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun