Some success and failure cases of our method are shown in Figure 3. Our method has
trouble differentiating instances which are occluded and also very visually similar to each
other (Fig. 3, last two rows). The supplementary material includes a more detailed results
table, and more examples of success and failure cases.
5 Conclusion
We have presented a simple method that effectively leverages state-of-the-art semantic segmentation and object detection networks to perform the less widely studied task of instance
segmentation. Our approach begins with a bottom-up semantic segmentation of the input
image, but is able to reason about instances since the ﬁnal CRF layer in our semantic segmentation network incorporates information from an object detector in the form of a higherorder, detection potential. Our method produces state-of-the-art instance segmentation results, achieving considerable improvements over existing work at high APrthresholds. Our
ﬁnal Instance CRF – which is dynamic in its number of labels – is fully differentiable. This
means that our neural network can be dynamically instantiated for every input image and
trained end-to-end.
Acknowledgements
We thank Bernardino Romera-Paredes, Stuart Golodetz and Shuai Zheng for insightful discussions and feedback. This work was supported by the EPSRC, Clarendon Fund, ERC
Traditional Scene Understanding problems such as Object Detection and
Semantic Segmentation have made breakthroughs in recent years due to the
adoption of deep learning. However, the former task is not able to localise
objects at a pixel level, and the latter task has no notion of different
instances of objects of the same class. We focus on the task of Instance
Segmentation which recognises and localises objects down to a pixel level. Our
model is based on a deep neural network trained for semantic segmentation. This
network incorporates a Conditional Random Field with end-to-end trainable
higher order potentials based on object detector outputs. This allows us to
reason about instances from an initial, category-level semantic segmentation.
Our simple method effectively leverages the great progress recently made in
semantic segmentation and object detection. The accurate instance-level
segmentations that our network produces is reflected by the considerable
improvements obtained over previous work.
Anurag Arnab Philip H. S. Torr