5 Experimental Results
In this section, we evaluate our approach for video object detection on the ImageNet VID [26] dataset, which has 3;862and555training and testing video
clips respectively. Each video is annotated with bounding boxes. The frames
from each video are extracted at 2530fps. The dataset contains 30object
categories that are a subset of the 200categories in the ImageNet DET dataset.
5.1 Quantitative Results
To assess the eﬀectiveness of our method we compare it to several relevant baselines, mainly two state-of-the-art methods FGFA [24] and D&T [27]. First, to
verify that using temporal information from video is beneﬁcial, we include a
static image-level variant of our model (SSN) that uses only the reference frame
Object Detection in Video with Spatiotemporal Sampling Networks 9
Methods
D&T [27] Our SSN FGFA [24] Our STSN D&T+ [27] Our STSN+
No FlowNet? 3 - 7 3 3 3
Not Using Flow Data? 3 - 7 3 3 3
No Temporal Post-Processing? 3 - 3 3 7 7
mAP@0.5 75.8 76.0 78.8 78.9 79.8 80.4
We propose a Spatiotemporal Sampling Network (STSN) that uses deformable
convolutions across time for object detection in videos. Our STSN performs
object detection in a video frame by learning to spatially sample features from
the adjacent frames. This naturally renders the approach robust to occlusion or
motion blur in individual frames. Our framework does not require additional
supervision, as it optimizes sampling locations directly with respect to object
detection performance. Our STSN outperforms the state-of-the-art on the
ImageNet VID dataset and compared to prior video object detection methods it
uses a simpler design, and does not require optical flow data for training.
Gedas Bertasius Lorenzo Torresani Jianbo Shi