separated by a line of low energy. This is seen in the rightmost vehicle in the 11th image. We anticipate that a combination of our method with top-down reasoning approaches
will greatly alleviate these two issues.
Because we rely upon correct semantic segmentation, errors from this (such as identifying a train as a bus) cannot be
ﬁxed by our method. This is clearly shown by the truck in
the last example. A possible solution could be to use semantic segmentation as soft gating, or to reason about semantic
and instance segmentation jointly. Finally, some very complex scenes such as some subgroups of people on the left in
the second to fourth example are incorrectly separated by
our model, and are fused together.
6. Conclusion
In this paper, we have proposed a simple instance segmentation technique inspired by the intuitive and classical watershed transform. Using a novel deep convolutional neural network and innovative loss functions for pretraining and ﬁne-tuning, we proposed a model that generates a modiﬁed watershed energy landscape. From this energy landscape, we directly extract high quality object instances. Our experiments show that we can more than double the performance of the state-of-the-art in the challenging Cityscapes Instance Segmentation task. We will release
the network weights and code for training and testing our
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun