shown to excel in the task of image classiﬁcation [19, 32], were adapted to object detection. Recent detection algorithms are based on using CNNs to classify object proposals
and then performing bounding box regression to reﬁne the initial proposals’ bounding boxes
[10, 11, 12, 28]. This pipeline can be accelerated by computing convolutional features over
the whole image, and then pooling features for each proposal [11, 15]. Object proposals
were initially hand-crafted [1, 34], but this step has now been integrated into CNNs [28].
Instance Segmentation: This task was popularised by Hariharan et al. [13]. Their approach was similar to object detection pipelines in that region proposals were ﬁrst generated and classiﬁed into different object categories before using bounding box regression as
post-processing. A class-speciﬁc segmentation was then performed to simultaneously detect
and segment an object. However, approaches that segment instances by reﬁning detections
[4, 5, 13, 14] are inherently limited by the quality of the initial proposals, and the systems
cannot recover from errors made by the initial detector. Furthermore, since these methods
generate and classify multiple overlapping region proposals per image, they cannot actually
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr