proposed in [ 54], but has not been used since in evaluating
instance segmentation systems.
4.3. Effect of Instance Potentials and End-to-End
training
We ﬁrst perform ablation studies on the VOC 2012 Validation set. This dataset, consisting of 1464 training and
1449 validation images has very high-quality annotations
with detailed object delineations which makes it the most
suited for evaluating pixel-level segmentations.
In Tab. 1, we examine the effect of each of our unary potentials in our Instance subnetwork on overall performance.
Furthermore, we examine the effect of end-to-end training
the entire network as opposed to piecewise training. Piecewise training refers to freezing the pretrained semantic segmentation subnetwork’s weights and only optimising the instance segmentation subnetwork’s parameters. Note that
when training with only the “Box” (Eq. 3) unary potential and pairwise term, we also have to add in an additional
“Background” detection which encompasses the entire image. Otherwise, we cannot classify the background label.
We can see that each unary potential improves overall instance segmentation results, both in terms of APr
voland the
Matching IoU. The “Global” term (Eq. 4) shows particular
Semantic segmentation and object detection research have recently achieved
rapid progress. However, the former task has no notion of different instances
of the same object, and the latter operates at a coarse, bounding-box level. We
propose an Instance Segmentation system that produces a segmentation map where
each pixel is assigned an object class and instance identity label. Most
approaches adapt object detectors to produce segments instead of boxes. In
contrast, our method is based on an initial semantic segmentation module, which
feeds into an instance subnetwork. This subnetwork uses the initial
category-level segmentation, along with cues from the output of an object
detector, within an end-to-end CRF to predict instances. This part of our model
is dynamically instantiated to produce a variable number of instances per
image. Our end-to-end approach requires no post-processing and considers the
image holistically, instead of processing independent proposals. Therefore,
unlike some related work, a pixel cannot belong to multiple instances.
Furthermore, far more precise segmentations are achieved, as shown by our
state-of-the-art results (particularly at high IoU thresholds) on the Pascal
VOC and Cityscapes datasets.
Anurag Arnab Philip H. S Torr