be explained by the fact that current semantic segmentation
systems (including ours) struggle with these classes. All recent methods on the Pascal VOC leaderboard2obtain an
IoU for these classes which is lower than the mean IoU for
all classes. In fact the semantic segmentation IoU for the
“chair” class is less than half of the mean IoU for all the
classes for 16 out of the 20 most recent submissions on the
VOC leaderboard at the time of writing.
Tables 7to9show per-class instance segmentation results on the VOC dataset, at IoU thresholds of 0.9, 0.7 and
0.5 respectively. At an IoU threshold of 0.9, our method
achieves the highest APrfor 16 of the 20 object classes.
At the threshold of 0.7, we achieve the highest APrin 15
classes. Finally, at an IoU threshold of 0.5, our method,
MPA 3-scale [ 37] and PFN [ 33] each achieve the highest
APrfor 6 categories.
B. Detailed results on the SBD dataset
Once again, we show a visualisation of the AProbtained
by our method for each class across nine different thresholds (Fig. 13). The trend is quite similar to the VOC dataset
in that our algorithm struggles on the same object classes
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr