features, and produce solid video object detection results even without temporal
post-processing algorithms that link bounding box detections over time.
Afterwards, we show that integrating a simple temporal post-processing algorithm Seq-NMS [22] further improves our STSN’s results. Such a scheme allows
us to outperform the D&T+ baseline ( 80:4vs79:8), which uses a similar Viterbi
based temporal post-processing scheme.
5.2 Ablation Studies
Optimal Number of Supporting Frames. In the left subplot of Figure 3,
we also illustrate how the number of supporting frames aﬀects the video object
detection accuracy. We notice that the performance keeps increasing as we add
more supporting frames, and then plateaus at T= 27.
Increasing the Temporal Stride. We also investigate how the temporal
stridek, at which we sample the supporting frames, aﬀects STSN’s performance.
We report that temporal strides of k= 2andk= 4, yield mAP scores of 79:0
and77:9, respectively. Thus, k= 2yields a slight improvement over our original
78:9mAP score. However, increasing kto larger values reduces the accuracy.
Feature Aggregation Weight Analysis. To analyze how much each of
We propose a Spatiotemporal Sampling Network (STSN) that uses deformable
convolutions across time for object detection in videos. Our STSN performs
object detection in a video frame by learning to spatially sample features from
the adjacent frames. This naturally renders the approach robust to occlusion or
motion blur in individual frames. Our framework does not require additional
supervision, as it optimizes sampling locations directly with respect to object
detection performance. Our STSN outperforms the state-of-the-art on the
ImageNet VID dataset and compared to prior video object detection methods it
uses a simpler design, and does not require optical flow data for training.
Gedas Bertasius Lorenzo Torresani Jianbo Shi