2012 [8] dataset, using the standard APrmeasure [13]. The fact that our system is able to
produce accurate instance segmentations is reﬂected by the considerable improvements that
we make over previous work at high overlap thresholds.
2 Related Work
Instance segmentation is closely related to the widely-studied problems of semantic segmentation and object detection. We thus review these two tasks ﬁrst.
Semantic Segmentation: This task was traditionally performed by extracting dense features from an image and classifying them on a per-pixel or per-region basis [16, 31]. The
individual predictions made by these classiﬁers were often noisy as they lacked global context. As a result, they were post-processed with a CRF whose priors encourages nearby
pixels, and pixels of similar appearance, to share the same class label. The CRF of [31]
initially contained only unary terms (obtained from the classiﬁer) and 8-connected pairwise
terms. Subsequent improvements included introducing densely-connected pairwise potentials [18], using higher order potentials encouraging consistency over cliques larger than two
pixels [17, 35], modelling the co-occurrence of labels [20, 35] and utilising the results of
object detectors [21, 33, 36].
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr