could have the same ﬁnal result. The works of [ 57] and [ 58]
order instances by depth to break this symmetry. However,
this requires ground-truth depth maps during training which
we do not assume that we have. Proposal-based methods
[12,19,20,37] do not have this issue since they consider a
single proposal at a time, rather than the entire image. Our
approach is similar to [ 45] in that we match the original
ground truth to our instance segmentation prediction based
on the Intersection over Union (IoU) [ 14] of each instance
prediction and ground truth, as shown in Fig. 6.
More formally, we denote the ground-truth labelling of
an image, G, to be a set of rsegments, {g1,g2,...,g r},
where each segment (set of pixels) is an object instance
and has an associated semantic class label. Our prediction, which is the output of our network, P, is a set of s
segments, {p1,p2,...,p s}, also where each segment corresponds to an instance label and also has an associated class
label. Note that randsmay be different since we may predict greater or fewer instances than actually present. Let
Mdenote the set of all permutations of the ground-truth,
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr