results on YouTube-VIS dataset. In future, we plan to extend MaskProp to scenarios where only bounding box annotations are available. We are also interested in applying our
method to problems such as pose estimation and tracking.
Video ClipMaskPropMaskTrack      R-CNN
Video ClipMaskPropMaskTrack      R-CNN
Video Clip
MaskProp
MaskTrack      R-CNNVideo Clip
MaskPropMaskTrack      R-CNN
Figure 7: We compare our video instance segmentation results with MaskTrack R-CNN [42] predictions. Different object
instances are encoded with different colors. The ﬁrst row for each video shows the original frames. The second row illustrates
the mask predictions of MaskTrack R-CNN and the third row those obtained with our MaskProp. Compared to MaskTrack
R-CNN, our MaskProp tracks object instances more robustly even when they are occluded or overlap with each other.
References
[1] Anurag Arnab and Philip H. S. Torr. Pixelwise instance segmentation with a dynamically instantiated network. In 2017
IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 ,
pages 879–888, 2017. 2
[2] M. Bai and R. Urtasun. Deep watershed transform for instance segmentation. In 2017 IEEE Conference on Computer
We introduce a method for simultaneously classifying, segmenting and tracking
object instances in a video sequence. Our method, named MaskProp, adapts the
popular Mask R-CNN to video by adding a mask propagation branch that propagates
frame-level object instance masks from each video frame to all the other frames
in a video clip. This allows our system to predict clip-level instance tracks
with respect to the object instances segmented in the middle frame of the clip.
Clip-level instance tracks generated densely for each frame in the sequence are
finally aggregated to produce video-level object instance segmentation and
classification. Our experiments demonstrate that our clip-level instance
segmentation makes our approach robust to motion blur and object occlusions in
video. MaskProp achieves the best reported accuracy on the YouTube-VIS dataset,
outperforming the ICCV 2019 video instance segmentation challenge winner
despite being much simpler and using orders of magnitude less labeled data
(1.3M vs 1B images and 860K vs 14M bounding boxes).
Gedas Bertasius Lorenzo Torresani