a greater level of penalty, which encourages the network to
focus on boundary regions. We pre-train the network using
the ground truth semantic segmentation and ground truth
direction predictions as input, and the discretized ground
truth modiﬁed watershed transform as target. All weights
are initialized using Xavier initialization [10]. We train the
network for 25 epochs using the ADAM optimizer. A batch
size of 6, constant learning rate of 5e-4, and a L2 weight
penalty of 1e-6 are used.
End-to-end ﬁne-tuning: We cascaded the pre-trained
models for the DN and WTN and ﬁne-tuned the complete
model for 20 epochs using the RGB image and semantic
segmentation output of PSPNet as input, and the ground
truth distance transforms as the training target. We use a
batch size of 3, constant learning rate of 5e-6, and a L2
weight penalty of 1e-6.
4.4. Energy Cut and Instance Extraction
We cut our watershed transform output at energy level 1
(0 to 2 pixels from boundary) for classes with many smaller
objects ( person ,rider ,bicycle ,motorcycle ), and level 2 (3 to
4 pixels from boundary) for classes with larger objects ( car,
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun