ﬁrst task. Instance segmentation is particularly challenging in street scenes, where the scale of the objects can vary
tremendously. Furthermore, the appearance of objects are
affected by partial occlusions, specularities, intensity saturation, and motion blur. Solving this task will, however,
tremendously beneﬁt applications such as object manipulation in robotics, or scene understanding and tracking for
self-driving cars.
Current approaches generally use complex pipelines to
handle instance extraction involving object proposals [20,
22, 7], conditional random ﬁelds (CRF) [32, 33], large recurrent neural networks (RNN) [24, 23, 2], or template
matching [28]. In contrast, we present an exceptionally
simple and intuitive method that signiﬁcantly outperforms
the state-of-the-art. In particular, we derive a novel approach which brings together classical grouping techniques
and modern deep neural networks.
The watershed transform is a well studied method inmathematical morphology. Its application to image segmentation can be traced back to the 70’s [4, 3]. The idea
behind this transform is fairly intuitive. Any greyscale image can be considered as a topographic surface. If we ﬂood
this surface from its minima and prevent the merging of the
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun