We do not require any post-processing, such as the patch
aggregation of [ 37], “mask-voting” of [ 12], “superpixel
projection” of [ 19,20,30] or spectral clustering of [ 33].
The fact that we compute an initial semantic segmentation
means that we have some robustness to errors in the object detector (Fig. 3). Furthermore, we are not necessarily
limited by poorly localised object detections either (Fig. 4).
Our CRF model allows us to reason about the entire image
at a time, rather than consider independent object proposals,
as done in [ 19,20,12,37,30]. Although we do not train our
object detector jointly with the network, it also means that
our segmentation network and object detector do not succumb to the same failure cases. Moreover, it ensures that
our instance labelling does not “switch” often during training, which makes learning more stable. Finally, note that
although we perform mean ﬁeld inference of a CRF within
our network, we do not optimise the CRF’s likelihood, but
rather a cross-entropy loss (Sec 3.4).
4. Experimental Evaluation
Sections 4.1to4.6describe our evaluation on the Pascal VOC Validation Set [ 14] and the Semantic Boundaries
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr