Deep Watershed Transform for Instance Segmentation
Min Bai Raquel Urtasun
Department of Computer Science, University of Toronto
fmbai, urtasun g@cs.toronto.edu
Abstract
Most contemporary approaches to instance segmentation use complex pipelines involving conditional random
ﬁelds, recurrent neural networks, object proposals, or template matching schemes. In this paper, we present a simple yet powerful end-to-end convolutional neural network to
tackle this task. Our approach combines intuitions from the
classical watershed transform and modern deep learning to
produce an energy map of the image where object instances
are unambiguously represented as energy basins. We then
perform a cut at a single energy level to directly yield connected components corresponding to object instances. Our
model achieves more than double the performance over
the state-of-the-art on the challenging Cityscapes Instance
Level Segmentation task.
1. Introduction
Instance segmentation seeks to identify the semantic
class of each pixel as well as associate each pixel with a
physical instance of an object. This is in contrast with
semantic segmentation, which is only concerned with the
ﬁrst task. Instance segmentation is particularly challenging in street scenes, where the scale of the objects can vary
tremendously. Furthermore, the appearance of objects are
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun