this has no impact on the actual quality of proposed object
instances, which remain the same. This shows the necessity
of a different metric such as muCov [26] that can evaluate
segmentation-based approaches fairly.
Qualitative Results: Fig. 5 depicts visualizations of sample results on the validation set, which is not used as part of
training. It is evident that our model produces very high
quality instance segmentation results. In these results, predicted and ground truth object instances only share the same
color if they have greater than 50% IoU.Failure Modes: Our model has several weaknesses.
Some images in Fig. 5 demonstrate these cases. The ﬁrst
issue is that the current formulation of our method does not
handle objects that have been separated into multiple pieces
by occlusion. This is most obvious in the 3rd image from
the bottom in Fig. 5 as the far-right vehicle is bisected by the
street sign pole, and the bicycle in the right part of the image just above. The resulting pieces are not merged into one
component. This is a drawback of most bottom-up grouping approaches. The second issue are cases where two objects sharing an occlusion boundary are mostly but not fully
separated by a line of low energy. This is seen in the rightmost vehicle in the 11th image. We anticipate that a combination of our method with top-down reasoning approaches
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun