iψU
i(xi)+∑
i<jψP
i,j(xi,xj)+∑
dψDet
d(xd,yd)+∑
dψU
d(yd). (3)
3.3 Instance Identiﬁcation and Reﬁnement
Once we have a category-level segmentation of the image, each pixel still needs to be assigned to an object instance. We assume that each object detection represents a possible
instance. Since there are Dobject detections (where Dvaries for every image), and some
pixels do not belong to any object instance, but are part of the background, we have a labelling problem involving D+1 labels. Our set of labels, D, is thus{1,2,...D+1}.
A naïve solution to the problem of recovering instances from a category-level segmentation would be as follows: If a pixel falls within the bounding box of a detection, and its
category label agrees with the class predicted by the object detector, then it is assigned to the
instance represented by that detection. However, this method cannot deal with overlapping
bounding boxes which are typical when objects occlude one another (as in Figs. 1 and 2).
Instead, if a pixel falls within the bounding box of a detection, we assign the pixel to
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr