reasons globally about the image and also produces more precise segmentations due to the initial semantic segmentation
module which includes a differentiable CRF. Row 1 shows a case where MNC, which scores segment-based proposals, is
fooled by a false-positive detection and segments an imaginary human (yellow segment). Our method is robust to falsepositive detections due to the initial semantic segmentation module which does not have the same failure modes as the
detector. Rows 2, 3 and 4 show how MNC [ 12] cannot deal with poorly localised bounding boxes. The horizontal boundaries
of the red person in Row 2, and light-blue person in Row 4 correspond to the limits of the proposal processed by MNC. Our
method, in contrast, can segment “outside the detection bounding box” due to the global instance unary potential (Eq. 4). As
MNC does not reason globally about the image, it cannot handle cases of overlapping bounding boxes well, and produces
more instances than there actually are. The ﬁrst column shows the input image, and the results of our object detector which
are another input to our network. MNC does not use these detections, but does internally produce box-based proposals which
are not shown. Best viewed in colour.
Input image MNC [ 12] Ours Ground truth
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr