where each pixel is assigned both a semantic class and instance label. Our end-to-end trained network, which outputs a variable number of instances per input image, begins with an initial semantic segmentation module. The
following, dynamic part of the network, then uses information from an object detector and a Conditional Random
Field (CRF) model to distinguish different instances. This
approach is robust to false-positive detections, as well as
poorly localised bounding boxes which do not cover the
entire object, in contrast to detection-based methods to instance segmentation. Moreover, as it considers the entire
image when making predictions, it attempts to resolve occlusions between different objects and can produce segmentation maps as in Fig. 1without any post-processing.
Furthermore, we note that the Average Precision (AP)
metric [ 14] used in evaluating object detection systems, and
itsAPrvariant [ 19] used for instance segmentation, considers individual, potentially overlapping, object predictions in
isolation, as opposed to the entire image. To evaluate methods such as ours, which produce complete segmentation
maps and reason about occlusions, we also evaluate using
1
arXiv:1704.02386v1  [cs.CV]  7 Apr 2017
(a) Object Detection (b) Semantic Segmentation (c) Instance Segmentation
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr