Instead, if a pixel falls within the bounding box of a detection, we assign the pixel to
that instance with a probability proportional to the rescored detection (obtained from the
probability of the latent Yvariable after inference) and the semantic segmentation conﬁdence
for that class, as shown in Equation 4
Pr(vi=k) =/braceleftBigg1
Z(Y,Q)Qi(lk)Pr(Yk=1)ifi∈Bk
0 otherwise .(4)
Here, viis a multinomial random variable indicating the “identiﬁed instance” at pixel iand
takes on labels from D,Qi(l)is the output of the initial category-level segmentation stage
of our network and denotes the probability of pixel itaking the label l∈L, and Z(Y,Q)
normalises the probability in case there are multiple bounding boxes overlapping the same
pixel. For this formulation to be valid, we also add another detection, d0, denoting background pixels that are not overlapped by any detection. This then acts as the unary potentials
of another CRF with the energy:
E(v) =∑
iψU
i(vi)+∑
i<jψP
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr