pixels [17, 35], modelling the co-occurrence of labels [20, 35] and utilising the results of
object detectors [21, 33, 36].
Using CNNs for semantic segmentation can greatly improve the unary, per-pixel predictions, as shown by [27]. Chen et al. showed further improvements by post-processing the
ARNAB AND TORR: BOTTOM-UP INSTANCE SEGMENTATION 3
CNN output with a CRF [3]. Current state-of-the-art methods [2, 25, 37] incorporate inference of a CRF as layers within a deep network which perform the differentiable mean ﬁeld
inference algorithm. In our work, we extend the end-to-end trainable, higher-order, object
detection potentials proposed by [2] to the task of instance segmentation. Additionally, unlike the aforementioned works which always assume a ﬁxed number of labels in the CRF at
all times, our system includes a CRF where the number of labels varies per input image.
Object Detection: Part-based models [9] were popular before CNNs, which had been
shown to excel in the task of image classiﬁcation [19, 32], were adapted to object detection. Recent detection algorithms are based on using CNNs to classify object proposals
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr