backbone and detection networks as our MaskProp.
These baselines allow us to directly compare the effectiveness of our mask propagation scheme versus the propagation mechanisms employed by FlowNet2 and MaskTrack
R-CNN [42] methods. The results in Table 3 show that
MaskProp outperforms these baselines by a large margin.
Instance Track Clip Length. Due to occlusions, object instances in video may not be visible in some frames. If there
are occlusions separated by 2T0+ 2time-steps, we can use
T >T0to predict longer clip-level instance tracks. In Figure 6, we study video instance segmentation performance as
a function of instance track clip length (denoted as 2T+ 1
in the paper). Our results indicate that the best accuracy is
achieved when we use a clip length of 13, meaning that we
propagate instances to 6previous and 6subsequent frames.
High-Resolution Mask Reﬁnement. We also study the impact of our high-resolution mask reﬁnement, described in
Subsection 4.4. We report that removing this reﬁnement
causes a drop of 1:9%in video instance segmentation mAP.
Importance of Frame-Level Instance Masks. As described in our main draft, we use frame-level instance masks
We introduce a method for simultaneously classifying, segmenting and tracking
object instances in a video sequence. Our method, named MaskProp, adapts the
popular Mask R-CNN to video by adding a mask propagation branch that propagates
frame-level object instance masks from each video frame to all the other frames
in a video clip. This allows our system to predict clip-level instance tracks
with respect to the object instances segmented in the middle frame of the clip.
Clip-level instance tracks generated densely for each frame in the sequence are
finally aggregated to produce video-level object instance segmentation and
classification. Our experiments demonstrate that our clip-level instance
segmentation makes our approach robust to motion blur and object occlusions in
video. MaskProp achieves the best reported accuracy on the YouTube-VIS dataset,
outperforming the ICCV 2019 video instance segmentation challenge winner
despite being much simpler and using orders of magnitude less labeled data
(1.3M vs 1B images and 860K vs 14M bounding boxes).
Gedas Bertasius Lorenzo Torresani