We have presented an end-to-end instance segmentation
approach that produces intermediate semantic segmentations, and shown that ﬁnetuning for instance segmentation
improves our network’s semantic segmentations. Our approach differs from other methods which derive their architectures from object detection networks [ 12,37,20] in that
our approach is more similar to a semantic segmentation
network. As a result, our system produces more accurate
and detailed segmentations as shown by our substantial improvements at high APrthresholds. Moreover, our system
produces segmentation maps naturally, and in contrast to
other published work, does not require any post-processing.
Finally, our network produces a variable number of outputs,depending on the number of instances in the image. Our future work is to incorporate an object detector into the endto-end training of our system to create a network that performs semantic segmentation, object detection and instance
segmentation jointly. Possible techniques for doing this are
suggested by [ 25] and [ 39].
Acknowledgements We thank Bernardino RomeraParedes and Stuart Golodetz for insightful discussions
and feedback. This work was supported by the EPSRC,
Clarendon Fund, ERC grant ERC-2012-AdG 321162HELIOS, EPRSRC grant Seebibyte EP/M013774/1 and
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr