the semantic segmentation into an instance segmentation (c). The semantic segmentation,
in which a large number of pixels have been correctly labelled as “person”, does not tell us
how many people there are and where each person is in the image. The instance segmentation answers these questions. Also note how our method is not affected by the false-positive
“bottle” detections.
We present a different approach to instance segmentation, where we initially perform
a category-level, semantic segmentation of the image before reasoning about the different
instances in the scene. We can identify instances because our semantic segmentation network
incorporates a Higher Order Conditional Random Field (CRF) which uses cues from the
output of an object detector. This CRF is fully differentiable and can be inserted as a layer
of a neural network [2, 37]. As a result, it can be used as a simple extension to an existing
semantic segmentation network to perform the related task of instance segmentation.
Our simple, bottom-up method is able to effectively leverage the progress made by stateof-the-art semantic segmentation and object detection networks to perform the related task
of instance segmentation. We show this by evaluating our approach on the PASCAL VOC
2012 [8] dataset, using the standard APrmeasure [13]. The fact that our system is able to
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr