instance segmentation will further beneﬁt our method.
5.3. Qualitative Results
In Figure 7, we compare our predicted clip-level instance
tracks (last row of predictions for each clip) with the MaskTrack R-CNN predictions (ﬁrst row of predictions). We
use different colors to represent different object instances.
Our qualitative results suggest that our MaskProp produces
more robust and temporally coherent instance tracks than
MaskTrack R-CNN. Such differences in performance are
especially noticeable when a video contains large object
motion, occlusions, or overlapping objects.
In Figure 5, we also visualize instance-speciﬁc features
that are propagated from frame tto other frames in the
given video clip for two different object instances detected
in framet. Here, we show activations from a randomly selected feature channel. Based on these results, we observe
that our MaskProp reliably propagates features that are speciﬁc to each instance despite motion blur, object deformations and large variations in object appearance.
6. Conclusion
In this work, we introduced MaskProp, a novel architecture for video instance segmentation. Our method is conceptually simple, it does not require large amounts of labeled data for pre-training, and it produces state-of-the-art
We introduce a method for simultaneously classifying, segmenting and tracking
object instances in a video sequence. Our method, named MaskProp, adapts the
popular Mask R-CNN to video by adding a mask propagation branch that propagates
frame-level object instance masks from each video frame to all the other frames
in a video clip. This allows our system to predict clip-level instance tracks
with respect to the object instances segmented in the middle frame of the clip.
Clip-level instance tracks generated densely for each frame in the sequence are
finally aggregated to produce video-level object instance segmentation and
classification. Our experiments demonstrate that our clip-level instance
segmentation makes our approach robust to motion blur and object occlusions in
video. MaskProp achieves the best reported accuracy on the YouTube-VIS dataset,
outperforming the ICCV 2019 video instance segmentation challenge winner
despite being much simpler and using orders of magnitude less labeled data
(1.3M vs 1B images and 860K vs 14M bounding boxes).
Gedas Bertasius Lorenzo Torresani