subsequently reason about instances. Moreover, the fact that we used results from both
semantic segmentation and object detection makes sense since instance segmentation can be
regarded as being at the intersection of these two more common problems.
4 Experimental Results and Analysis
We ﬁrst describe the dataset and our experimental setup (§4.1), before examining the efﬁcacy
of our detection potentials (§4.2) and comparing our approach with other recent instance
segmentation methods (§4.3).
4.1 Experimental Setup
Following previous work, we evaluate our instance segmentation performance on the PASCAL VOC 2012 [8] validation set which comprises of 1449 images with high-quality annotations. There is no test server for instance segmentation to evaluate on the test set. We
use the standard APrmeasure for evaluation [13], which computes the mean average precision under different Intersection over Union (IoU) scores between the predicted and ground
truth segmentations (rather than IoU between bounding boxes, as in object detection). In object detection, a predicted bounding box is considered correct if the IoU between it and the
ground-truth bounding box is greater than 0.5. Here, we consider different overlap thresholds
between the predicted and ground-truth segments, since higher overlap thresholds require the
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr