ARNAB AND TORR: BOTTOM-UP INSTANCE SEGMENTATION 5
data-dependent partition function. From here onwards, we drop the conditioning on Ito
keep the notation uncluttered. The energy of an assignment is deﬁned over the set of cliques
Cin the graph Gas
E(x) =∑
c∈Cψc(xc), (1)
where xcis the vector formed by selecting elements of xthat correspond to random variables
belonging to the clique c, andψc(.)is the cost function for a clique c.
The energy function considered by many semantic segmentation works, E(x) =∑iψU
i(xi)+
∑i<jψP
i j(xi,xj), consists of only unary (cliques of size one) and densely-connected pairwise
potentials (cliques of size two) [3, 18, 37]. When dense pairwise potentials are used in a
CRF to obtain higher accuracy, exact inference is not tractable and approximate inference
methods such as mean ﬁeld are used [18].
3.2 Higher Order Detection Potentials
In addition to the standard unary and densely-connected pairwise terms, we also include a
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr