potentials [ 26] and encourages appearance and spatial consistency. The weights governing the importance of these
terms are also learnt via backpropagation, as in [ 60]. We
ﬁnd that these priors are useful in the case of instance segmentation as well, since nearby pixels that have similar appearance often belong to the same object instance. They are
often able to resolve occlusions based on appearance differences between objects of the same class (Fig. 3).
3.3. Inference of our Dynamic Instance CRF
We use mean ﬁeld inference to approximately minimise
the Gibbs Energy in Eq. 1which corresponds to ﬁnding the
Maximum a Posteriori (MAP) labelling of the corresponding probability distribution, P(V=v) =1
Zexp (−E(v))
whereZis the normalisation factor. Mean ﬁeld inference is
differentiable, and this iterative algorithm can be unrolled
and seen as a recurrent neural network [ 60]. Following this
approach, we can incorporate mean ﬁeld inference of a CRF
as a layer of our neural network. This enables us to train our
entire instance segmentation network end-to-end.
Because we deal with a variable number of instances for
every image, our CRF needs to be dynamically instantiated
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr