of75%, and 3) average recall given 1and10highest scored
instances per video. From these results, we observe that
our MaskProp outperforms all the other baselines according
to all four evaluation metrics, thus achieving state-of-theart results in video instance segmentation on YouTube-VIS.
It can be noted that we outperform EnsembleVIS [28] by
1:8mAP even though our approach is much simpler and
even if it uses orders of magnitude less labeled data for
pre-training. Furthermore, compared to the MaskTrack RCNN, our method achieves a 16:3improvement in mAP. We
also note that our implementation of MaskTrack [42] signiﬁcantly improves upon the original work, but it is still 9:7
Method mAP AP@75
FlowNet2 Propagation 31.4 33.6
MaskTrack R-CNN36.9 40.2
MaskProp 46.6 51.2
Table 3: Here, we study the effectiveness of our mask propagation branch. If we replace it with the FlowNet2 propagation scheme, where masks are propagated using the optical
ﬂow predicted by a FlowNet2 network [20], the accuracy
drops from 46:6mAP to 31:4mAP. Similarly, if we replace
our mask propagation branch with the tracking branch from
We introduce a method for simultaneously classifying, segmenting and tracking
object instances in a video sequence. Our method, named MaskProp, adapts the
popular Mask R-CNN to video by adding a mask propagation branch that propagates
frame-level object instance masks from each video frame to all the other frames
in a video clip. This allows our system to predict clip-level instance tracks
with respect to the object instances segmented in the middle frame of the clip.
Clip-level instance tracks generated densely for each frame in the sequence are
finally aggregated to produce video-level object instance segmentation and
classification. Our experiments demonstrate that our clip-level instance
segmentation makes our approach robust to motion blur and object occlusions in
video. MaskProp achieves the best reported accuracy on the YouTube-VIS dataset,
outperforming the ICCV 2019 video instance segmentation challenge winner
despite being much simpler and using orders of magnitude less labeled data
(1.3M vs 1B images and 860K vs 14M bounding boxes).
Gedas Bertasius Lorenzo Torresani