at intersection over union (IoU) threshold of 0:5.
10 G. Bertasius, L. Torresani, J. Shi
0 2 6 10 14 18 22 26 30Number of Supporting Frames7676.57777.57878.579mAP
-10 -8 -6 -4 -2 0 2 4 6 8 10Time Delta w.r.t Reference Frame0.060.0650.070.0750.080.0850.090.0950.1Weight Magnitude
Fig.3: A ﬁgure illustrating some of our ablation experiments. Left:we plot mAP
as a function of the number of supporting frames used by our STSN. From this
plot, we notice that the video object detection accuracy improves as we use
more supporting frames. Right:To understand the contribution of each of the
supporting frames, we plot the average weight magnitudes wt;t+k(p)for diﬀerent
values ofk. Here,prepresents a point at the center of an object. From this plot,
we observe that the largest weights are associated with the supporting frames
that are near the reference frame. However, note that even supporting frames
that are further away from the reference frame (e.g. k= 9) contribute quite
substantially to the ﬁnal object detection predictions.
We propose a Spatiotemporal Sampling Network (STSN) that uses deformable
convolutions across time for object detection in videos. Our STSN performs
object detection in a video frame by learning to spatially sample features from
the adjacent frames. This naturally renders the approach robust to occlusion or
motion blur in individual frames. Our framework does not require additional
supervision, as it optimizes sampling locations directly with respect to object
detection performance. Our STSN outperforms the state-of-the-art on the
ImageNet VID dataset and compared to prior video object detection methods it
uses a simpler design, and does not require optical flow data for training.
Gedas Bertasius Lorenzo Torresani Jianbo Shi