label,si∈[0,1]is the conﬁdence score and Biis the set of
indices of the pixels falling within the detector’s bounding
box. Note that the number Dvaries for every input image.
The problem of instance segmentation can then be
thought of as assigning every pixel to either a particular object detection, or the background label. This is based on
the assumption that every object detection speciﬁes a potential object instance. We deﬁne a multinomial random
variable,V, at each of the Npixels in the image, and
V= [V1V2...V N]T. Each variable at pixel i,Vi, is assigned a label corresponding to its instance. This label set,
{0,1,2,...,D }changes for each image since D, the number
of detections, varies for every image (0 is the background
label). In the case of instance segmentation of images, the
quality of a prediction is invariant to the permutations of the
instance labelling. For example, labelling the “blue person”
in Fig. 1(c) as “1” and the “purple person” as “2” is no different to labelling them as “2” and “1” respectively. This
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr