D&T [27] and STSN refer to the CNN-level baselines whereas D&T+ [27] and
STSN+ denote these same methods but with temporal post-processing (i.e. SeqNMS[22],object-tubebasedlinking[27],etc)appliedontopoftheCNNoutputs.
We present our results in Table 1, where we assess each method according to
several criteria. In the ﬁrst row of Table 1, we list whether a given method requires integrating a separate ﬂow network into its training / prediction pipeline.
Ideally, we would want to eliminate this step because optical ﬂow prediction requiresdesigningahighlycomplexﬂownetworkarchitecture.Wealsolistwhether
a given method requires pre-training on the external optical ﬂow data, which we
would want to avoid since it makes the whole training pipeline more costly.
Additionally, we list, whether a given method uses any external temporal postprocessing steps, which we would want to eliminate because they typically make
the training / prediction pipeline disjoint and more complex. Finally, we assess
each method according to the standard mean average precision (mAP) metric
at intersection over union (IoU) threshold of 0:5.
10 G. Bertasius, L. Torresani, J. Shi
We propose a Spatiotemporal Sampling Network (STSN) that uses deformable
convolutions across time for object detection in videos. Our STSN performs
object detection in a video frame by learning to spatially sample features from
the adjacent frames. This naturally renders the approach robust to occlusion or
motion blur in individual frames. Our framework does not require additional
supervision, as it optimizes sampling locations directly with respect to object
detection performance. Our STSN outperforms the state-of-the-art on the
ImageNet VID dataset and compared to prior video object detection methods it
uses a simpler design, and does not require optical flow data for training.
Gedas Bertasius Lorenzo Torresani Jianbo Shi