cannot recover from errors made by the initial detector. Furthermore, since these methods
generate and classify multiple overlapping region proposals per image, they cannot actually
produce segmentation maps of the form shown in Figures 1(b) and (c) [24]. Liang et al. [23]
address some of these shortcomings by iteratively reﬁning initial object proposals, whilst [7]
generates box-proposals, creates masks from these proposals and then classiﬁes these masks
in a single network trained with three loss functions summed together.
A proposal-free method was recently presented by [24] where a semantic segmentation of the image is ﬁrst performed using the network of [3]. Thereafter the category-level
segmentation output, along with CNN features, are used to predict instance-level bounding
boxes for the objects in the image. The number of instances of each object category are also
predicted in order to facilitate the ﬁnal clustering step. An alternate method was proposed
by [29] where a Recurrent Neural Network outputs an object instance at each time step. This
method, however, has not been evaluated on multiple classes.
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr