(a) Object Detection (b) Semantic Segmentation (c) Instance Segmentation
Figure 1: Object detection (a) localises the different people, but at a coarse, bounding-box level. Semantic segmentation
(b) labels every pixel, but has no notion of instances. Instance segmentation (c) labels each pixel of each person uniquely.
Our proposed method jointly produces both semantic and instance segmentations. Our method uses the output of an object
detector as a cue to identify instances, but is robust to false positive detections, poor bounding box localisation and occlusions.
Best viewed in colour.
the “Matching Intersection over Union” metric.
Our system, which is based on an initial semantic segmentation subnetwork, produces sharp and accurate instance segmentations. This is reﬂected by the substantial improvements we achieve over state-of-the-art methods
at highAPrthresholds on the Pascal VOC and Semantic
Boundaries datasets. Furthermore, our network improves
on the semantic segmentation task while being trained for
the related task of instance segmentation.
2. Related Work
An early work on instance segmentation was by Winn
and Shotton [ 51]. A per-pixel unary classiﬁer was trained
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr