[19], whilst [ 32] iteratively reﬁned object proposals.
On a separate track, algorithms have also been developed
that do not require object detectors. Zhang et al. [57,58]
segmented car instances by predicting the depth ordering
of each pixel in the image. Unlike the previous detectionbased approaches, this method reasoned globally about all
instances in the image simultaneously (rather than individual proposals) with an MRF-based formulation. However,
inference of this graphical model was not performed end-toend as shown to be possible in [ 60,2,5,34]. Furthermore,
although this method does not use object detections, it is
trained with ground truth depth and assumes a maximum
of nine cars in an image. Predicting all the instances in
an image simultaneously (rather than classifying individual
proposals) requires a model to be able to handle a variable
number of output instances per image. As a result, [ 45]
proposed a Recurrent Neural Network (RNN) for this task.
However, this model was only for a single object category.
Our proposed method not only outputs a variable number of
instances, but can also handle multiple object classes.
Liang et al. [33] developed another proposal-free method
based on the semantic segmentation network of [ 6]. The
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr