that are further away from the reference frame (e.g. k= 9) contribute quite
substantially to the ﬁnal object detection predictions.
Based on our results in Table 1, we make the following conclusions. First, we
notethatourSTSNproducesbetterquantitativeresultsthanthestate-of-the-art
FGFA method ( 78:9vs78:8). We acknowledge that our accuracy improvement
over FGFA is moderate. However, we point out that our STSN operates in a
much more challenging setting than FGFA. Unlike FGFA, our STSN does not
use any optical ﬂow supervision. Instead, it is trained directly for video object
detection. The fact that STSN learns temporal correspondences without direct
optical ﬂow supervision, and still outperforms FGFA is quite impressive. Such
results also show the beneﬁt of discriminative end-to-end training with respect
to the ﬁnal video object detection task objective.
WenextcompareourSTSNtotheD&Tbaseline[27].Wenotethatunlikefor
the FGFA [24] baseline, it is much harder to make a direct comparison between
We propose a Spatiotemporal Sampling Network (STSN) that uses deformable
convolutions across time for object detection in videos. Our STSN performs
object detection in a video frame by learning to spatially sample features from
the adjacent frames. This naturally renders the approach robust to occlusion or
motion blur in individual frames. Our framework does not require additional
supervision, as it optimizes sampling locations directly with respect to object
detection performance. Our STSN outperforms the state-of-the-art on the
ImageNet VID dataset and compared to prior video object detection methods it
uses a simpler design, and does not require optical flow data for training.
Gedas Bertasius Lorenzo Torresani Jianbo Shi