behind this transform is fairly intuitive. Any greyscale image can be considered as a topographic surface. If we ﬂood
this surface from its minima and prevent the merging of the
waters coming from different sources, we effectively partition the image into different components (i.e., regions).
This transformation is typically applied to the image gradient, thus the basins correspond to homogeneous regions in
the image. A signiﬁcant limitation of the watershed transform is its propensity to over-segment the image. One of
the possible solutions is to estimate the locations of object
instance markers, which guide the selection of a subset of
these basins [11, 21]. Heuristics on the relative depth of the
basins can be exploited in order to merge basins. However,
extracting appropriate markers and creating good heuristics
is difﬁcult in practice. As a consequence, modern techniques for instance segmentation do not exploit the watershed transform.
In this paper, we propose a novel approach which combines the strengths of modern deep neural networks with the
power of this classical bottom-up grouping technique. We
propose to directly learn the energy of the watershed transform such that each basin corresponds to a single instance,
while all dividing ridges are at the same height in the energy domain. As a consequence, the components can be
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun