but ignore the latent Yvariables output by the CRF and use the detector’s original conﬁdence
score instead as the input to the instance segmentation network. The second row of Table
1 shows that this second baseline has an APrat 0.5 and an APr
volthat are respectively 0.8%
and 0.7% lower than our ﬁnal method. The difference between the ﬁnal method and this
second baseline is due to the score recalibration performed by the detection potentials. The
two baselines differ in performance since the second baseline uses a semantic segmentation
(mean IoU of 75.3%) which is better than the other (73.4%) due to its use of detection
potentials. The large gap in their overall instance segmentation performance emphasises
the importance of a good initial semantic segmentation for our method, and underlines the
beneﬁts of a bottom-up approach to instance segmentation.
From the difference in performance between the baseline that does not fully make use of
detection potentials (Row 1 of Table 1), and our ﬁnal method (Row 3 of Table 1), we can
conclude that our higher order detection potentials are an integral part of our system.
4.3 Comparison to other current methods
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr