Many recent approaches to instance segmentation arebased on object detection pipelines where objects are ﬁrst
localised with bounding boxes. Thereafter, each bounding
box is reﬁned into a segmentation [ 19,20,32,37,30]. Another related approach [ 12,56] is to use segment-based region proposals [ 10,41,42] instead of box-based proposals.
However, these methods do not consider the entire image,
but rather independent proposals. As a result, occlusions
between different objects are not handled. Furthermore,
many of these methods cannot easily produce segmentation
maps of the image, as shown in Fig. 1, since they process
numerous proposals independently. There are typically far
more proposals than actual objects in the image, and these
proposals can overlap and be assigned different class labels.
Finally, as these methods are based on an initial detection
step, they cannot recover from false detections.
Our proposed method is inspired by the fact that instance
segmentation can be viewed as a more complex form of semantic segmentation, since we are not only required to label the object class of each pixel, but also its instance identity. We produce a pixelwise segmentation of the image,
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr