of0:008, which is decreased by 10at16and19epochs.
We initialize our model with a Mask R-CNN pretrained on
COCO for the instance segmentation. The hyperparameters
of RPN and FPN are the same as in [7].
Inference. During testing, we run the bounding box prediction branch on 1000 proposals, apply non-maximum suppression, and use boxes with a score higher than 0:1as input to the mask prediction and mask propagation branches.
During inference, our MaskProp is applied to video clips
consisting of 13frames.
5. Experimental Results
In this section, we evaluate MaskProp for video instance
segmentation on YouTube-VIS [42], which contains 2;238
training, 302validation, and 343test videos. Each video
is annotated with per-pixel segmentation, category, and instance labels. The dataset contains 40object categories.
Since the evaluation on the test set is currently closed, we
perform our evaluations on the validation set.
5.1. Quantitative Results
Video instance segmentation is a very recent task [42],
and thus, there are only a few established baselines that we
7 9 11 13 15 17 19 21
We introduce a method for simultaneously classifying, segmenting and tracking
object instances in a video sequence. Our method, named MaskProp, adapts the
popular Mask R-CNN to video by adding a mask propagation branch that propagates
frame-level object instance masks from each video frame to all the other frames
in a video clip. This allows our system to predict clip-level instance tracks
with respect to the object instances segmented in the middle frame of the clip.
Clip-level instance tracks generated densely for each frame in the sequence are
finally aggregated to produce video-level object instance segmentation and
classification. Our experiments demonstrate that our clip-level instance
segmentation makes our approach robust to motion blur and object occlusions in
video. MaskProp achieves the best reported accuracy on the YouTube-VIS dataset,
outperforming the ICCV 2019 video instance segmentation challenge winner
despite being much simpler and using orders of magnitude less labeled data
(1.3M vs 1B images and 860K vs 14M bounding boxes).
Gedas Bertasius Lorenzo Torresani