our approach. Despite its simplicity, MaskProp outperforms all prior video instance segmentation methods by a large margin.
Scoring Video-Level Sequences. Each video-level sequence contains a list of classiﬁcation scores and predicted
object labels. To assign a conﬁdence score to each videolevel sequence, we average classiﬁcation scores associated
with that sequence (separately for each object category).
Training. We use a similar training setup as in [7]. The loss
weights for each of the three cascade stages are set to 1;0:5;
and0:25respectively. The loss weight for the semantic segmentation branch is set to 0:1. We train our model on pairs
of frames, where the second frame in a pair is randomly selected with a time gap 2[25;25]relative to ﬁrst frame.
We use a multi-scale training approach implemented by resizing the shorter side of the frame randomly between 400
and800pixels. Our model is trained in a distributed setting using 64GPUs, each GPU holding a single clip. The
training is done for 20epochs with an initial learning rate
of0:008, which is decreased by 10at16and19epochs.
We initialize our model with a Mask R-CNN pretrained on
We introduce a method for simultaneously classifying, segmenting and tracking
object instances in a video sequence. Our method, named MaskProp, adapts the
popular Mask R-CNN to video by adding a mask propagation branch that propagates
frame-level object instance masks from each video frame to all the other frames
in a video clip. This allows our system to predict clip-level instance tracks
with respect to the object instances segmented in the middle frame of the clip.
Clip-level instance tracks generated densely for each frame in the sequence are
finally aggregated to produce video-level object instance segmentation and
classification. Our experiments demonstrate that our clip-level instance
segmentation makes our approach robust to motion blur and object occlusions in
video. MaskProp achieves the best reported accuracy on the YouTube-VIS dataset,
outperforming the ICCV 2019 video instance segmentation challenge winner
despite being much simpler and using orders of magnitude less labeled data
(1.3M vs 1B images and 860K vs 14M bounding boxes).
Gedas Bertasius Lorenzo Torresani