learnable weight parameter. This potential encourages consistency among X(i)
dvariables and
6 ARNAB AND TORR: BOTTOM-UP INSTANCE SEGMENTATION
Ydsince it encourages X(i)
dvariables to take the label ldwhen Ydis 1, and also encourages Yd
to be 0 when many X(i)
dvariables do not take the label ld.
A unary potential for the latent Yvariables is also included, which is initialised to the
conﬁdence score sdof the object detector (in addition to the joint potential with the Xvariables). This initial conﬁdence then changes throughout CRF inference, depending on the
other potentials. The ﬁnal probability of the latent Yvariable can thus be viewed as a recalibrated detection score, which gets adjusted during inference based on how much it agrees
with the segmentation unaries and pairwise potentials. Note that this “recalibration property”
of the latent Yvariables was not considered by [2]. The ﬁnal form of the energy function
(Eq 1), is thus
E(x) =∑
iψU
i(xi)+∑
i<jψP
i,j(xi,xj)+∑
dψDet
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr