label of the detected object, sdis the detector’s conﬁdence score, Fd⊂{1,2,...N}is the set
of indices of the pixels belonging to the foreground of the detection and Bd⊂{1,2,...N}is
the set of indices falling in the detection’s bounding box. The foreground is obtained from
a foreground/background segmentation method such as GrabCut [30] and indicates a rough
segmentation of the detected object. The detection potentials should encourage the set of
pixels represented by Fdto take the object class label, ld. However, this should be a soft
constraint, since the foreground segmentation could be inaccurate and the entire detection
itself could be a false-positive. The detection dshould also be identiﬁed as invalid if other
energies in the CRF strongly suggest that many pixels in Fddo not take the label ld.
This is formulated by introducing a latent binary random variable, Y1,Y2...YDfor every
detection. If the dthdetection has been found to be valid after inference, Ydwill be set to 1,
and 0 otherwise. The ﬁnal value of Ydis determined probabilistically by mean ﬁeld inference.
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr