4. Experimental Evaluation
Sections 4.1to4.6describe our evaluation on the Pascal VOC Validation Set [ 14] and the Semantic Boundaries
Dataset (SBD) [ 18] (which provides per-pixel annotations
to 11355 previously unlaballed images from Pascal VOC).
Section 4.7details results on Cityscapes [ 9].
4.1. Experimental Details
We ﬁrst train a network for semantic segmentation, therafter we ﬁnetune it to the task of instance segmentation,
as described in Sec. 3.5. Our training data for the semantic segmentation pretraining consists of images from Pascal VOC [ 14], SBD [ 18] and Microsoft COCO [ 35]. Finally, when ﬁnetuning for instance segmentation, we use
only training data from either the VOC dataset, or from the
SBD dataset. We train separate models for evaluating on
the VOC Validation Set, and the SBD Validation Set. In
each case, we remove validation set images from the initial
semantic segmentation pretraining set. We use the publicly
available R-FCN object detection framework [ 13], and ensure that the images used to train the detector do not fall
into our test sets for instance segmentation.
4.2. Evaluation Metrics
We report the mean Average Precision over regions
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr