both [13] and [4] which initially perform object detection, and reﬁne these detections into an
instance segmentation. In fact, even our baselines in Table 1 outperform them. The model
of [24] differs from the other two works in that the ﬁrst stage of their pipeline is to also
perform a category-level segmentation of the image. However, their method does not make
use of any information from object detectors, and our approach performs better, particularly
at high IoU thresholds where the difference in APrat an IoU threshold of 0.9 is 4.4%. This
indicates that our instance segmentations are more precise. The fact that our APr
volis only
0.8% greater, despite our APrbeing signiﬁcantly larger at high thresholds suggests that the
method of [24] has a higher APrat lower thresholds (below 0.5, which were not reported by
[24]). This indicates that their method identiﬁes more instances than ours, but the fact that
their APrat higher thresholds is lower than ours also suggests that their segmentations are
not as accurate.
Some success and failure cases of our method are shown in Figure 3. Our method has
trouble differentiating instances which are occluded and also very visually similar to each
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr