loss:
ldirection =X
p2P objwpkcos1< ~ up;GT; ~ up;pred>k2
wherePobjis the set of all pixels belonging to a semantic class with object instances, and wpis a weighting factor proportional to the inverse of the square root of the object instance’s area. We use the mean squared angular error as opposed to the commonly used cosine distance, as
large angular errors (occuring when the network associates
a pixel with the wrong object instance) should incur significant larger penalties than small angular errors (which have
little impact).
This network is pre-trained using the original input images gated by the ground truth semantic segmentation and
the ground truth unit vectors ~ uGT. The lower layers (conv1
to conv5) are initialized with VGG16 [27]. The weights of
the upper layers are initialized randomly according to the
Xavier initialization scheme [10]. However, the intialization variance of the weights of the ﬁnal layer of each prediction branch is set manually such that the variance of each
branch’s output is of the same order of magnitude before
concatenation. This encourages the network to consider the
output of each branch. We use the ADAM optimizer to train
Most contemporary approaches to instance segmentation use complex pipelines
involving conditional random fields, recurrent neural networks, object
proposals, or template matching schemes. In our paper, we present a simple yet
powerful end-to-end convolutional neural network to tackle this task. Our
approach combines intuitions from the classical watershed transform and modern
deep learning to produce an energy map of the image where object instances are
unambiguously represented as basins in the energy map. We then perform a cut at
a single energy level to directly yield connected components corresponding to
object instances. Our model more than doubles the performance of the
state-of-the-art on the challenging Cityscapes Instance Level Segmentation
task.
Min Bai Raquel Urtasun