the network weights and code for training and testing our
model. In the future, we plan to augment the method to handle object instances bisected by occlusions. Additionally,
we wish to explore the possibility of extending our approach
to perform joint semantic and instance level segmentation,
in hopes of further reﬁning both outputs simultaneously.
Acknowledgements: This work was partially supported
by ONR-N00014-14-1-0232, Samsung, NVIDIA, Google
and NSERC.
(a) Input Image
 (b) Sem. Segmentation [34]
 (c) Our Instance Segmentation
 (d) GT Instance Segmentation
Figure 5: Sample output of our model on the validation set. Note that predicted object instances and ground truth object
instances are only given the same color if they have over 50% IoU.
References
[1] P. Arbelaez, J. Pont-Tuset, J. Barron, F. Marques, and J. Malik. Multiscale Combinatorial Grouping. In CVPR , 2014.
2
[2] A. Arnab and P. Torr. Bottom-up instance segmentation using deep higher-order crfs. In BMVC , 2016. 1, 2
[3] S. Beucher. The watershed transformation applied to image segmentation. Scanning Microscopy International ,
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun