associate these pixels with. Using heuristics, we could propose additional detections in cases like these. However, we have
not done this in our work. Third row: A difﬁcult case where we have segmented most of the people. However, sometimes
two people instances are joined together as one person instance. This problem is because we do not have a detection for
each person in the image. Fourth row: Due to our initial semantic segmentation, we have not been able to segment the green
person and table correctly. Fifth row: We have failed to segment a bird although it was detected. Sixth row: The occluding
cows, which all appear similar, pose a challenge, even with our shape priors. The ﬁrst column shows the input image, and
the results of our object detector which are another input to our network. Best viewed in colour.
Input image MNC [ 12] Ours Ground truth
Figure 9: Comparison to MNC [ 12]The above examples emphasise the advantages in our method over MNC [ 12]. Unlike
proposal-based approaches such as MNC, our method can handle false-positive detections, poor bounding box localisation,
reasons globally about the image and also produces more precise segmentations due to the initial semantic segmentation
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr