condition is handled by our loss function in Sec. 3.4.
Note that unlike works such as [ 57] and [ 58] we do not
assume a maximum number of possible instances and keep
a ﬁxed label set. Furthermore, since we are considering object detection outputs jointly with semantic segmentation
predictions, we have some robustness to high-scoring false
positive detections unlike methods such as [ 8,20,37] which
reﬁne object detections into segmentations.
We formulate a Conditional Random Field over our instance variables, V, which consists of unary and pairwise
(a) Only Box term (b) Box and Global terms
Figure 4: The “Global” unary potential (b) is particularly
effective in cases where the input detection bounding box
does not cover the entire extent of the object. Methods
which are based on reﬁning bounding-box detections such
as [19,20,8,12] cannot cope with poorly localised detections. Note, the overlaid detection boxes are an additional
input to our system.
energies. The energy of the assignment vto all the variables,V, is
E(V=v) =/summationdisplay
iU(vi) +/summationdisplay
i<jP(vi,vj). (1)
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr