Video instance segmentation is a very recent task [42],
and thus, there are only a few established baselines that we
7 9 11 13 15 17 19 21
Instance Track Clip Length43.54444.54545.54646.547Accuracy (Video mAP)Figure 6: We plot video mAP as a function of an instance
track clip length (denoted as 2T+ 1in the paper). Based on
these results, we observe that optimal video instance segmentation performance is achieved when we propagate instance masks to T= 6previous and subsequent frames.
can compare our work to. We include in our comparison
MaskTrack R-CNN [42], and the EnsembleVIS method,
which won the ICCV 2019 video instance segmentation
challenge [28]. Additionally, to make the comparison with
MaskTrack R-CNN more fair, we reimplement it using the
same backbone and detection networks as our MaskProp
(see MaskTrack R-CNN* in Table 2).
We present our quantitative results in Table 2, where we
assess each method according to 1) mean video average precision (mAP), 2) video average precision at IoU threshold
of75%, and 3) average recall given 1and10highest scored
instances per video. From these results, we observe that
our MaskProp outperforms all the other baselines according
We introduce a method for simultaneously classifying, segmenting and tracking
object instances in a video sequence. Our method, named MaskProp, adapts the
popular Mask R-CNN to video by adding a mask propagation branch that propagates
frame-level object instance masks from each video frame to all the other frames
in a video clip. This allows our system to predict clip-level instance tracks
with respect to the object instances segmented in the middle frame of the clip.
Clip-level instance tracks generated densely for each frame in the sequence are
finally aggregated to produce video-level object instance segmentation and
classification. Our experiments demonstrate that our clip-level instance
segmentation makes our approach robust to motion blur and object occlusions in
video. MaskProp achieves the best reported accuracy on the YouTube-VIS dataset,
outperforming the ICCV 2019 video instance segmentation challenge winner
despite being much simpler and using orders of magnitude less labeled data
(1.3M vs 1B images and 860K vs 14M bounding boxes).
Gedas Bertasius Lorenzo Torresani