(d) GT Instance Segmentation
Figure 1: Sample prediction: the input image is gated by sem. segmentation from [34] and passed through our model.
2. Related
Several instance level segmentation approaches have
been proposed in recent years. We now brieﬂy review them.
Proposal based: Many approaches are based on the reﬁnement of object proposals. For example, [1] generates
object segments proposals, and reasons about combining
them into object instances. In a similar spirit, [12] selects
proposals using CNN features and non-maximum suppression. Based on this, [5] further reasons about multiple object proposals to handle occlusion scenarios where single
objects are split into multiple disconnected patches. [7] uses
a deep cascaded neural network to propose object instance
bounding boxes, reﬁne instance masks, and semantically
label the masks in sequence. [20, 22] generate segmentation proposals using deep CNNs, which are then further reﬁned to achieve better segmentation boundaries. Additionally, [31] uses a modiﬁed R-CNN model to propose instance
bounding boxes, which are then further reﬁned to obtain instance level segmentation.
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun