No Temporal Post-Processing? 3 - 3 3 7 7
mAP@0.5 75.8 76.0 78.8 78.9 79.8 80.4
Table 1: We use the ImageNet VID [26] dataset to compare our STSN to the
state-of-the-art FGFA [24] and D&T [27] methods. Note that SSN refers to our
static baseline, which is obtained by using only the reference frame for output
generation (no temporal info). Also note, that D&T+ and STSN+ refer to D&T
and STSN baselines with temporal post-processing applied on top of the CNN
outputs. Based on these results, we ﬁrst point out that unlike FGFA, our STSN
does not rely on the external optical ﬂow data, and still yields higher mAP
(78:9vs78:8). Furthermore, when no temporal post-processing is used, our
STSN produces superior performance in comparison to the D&T baseline ( 78:9
vs75:8). Finally, we demonstrate that if we use a simple Seq-NMS [22] temporal
post-processing scheme on top of our STSN predictions, we can further improve
our results and outperform all the other baselines.
We propose a Spatiotemporal Sampling Network (STSN) that uses deformable
convolutions across time for object detection in videos. Our STSN performs
object detection in a video frame by learning to spatially sample features from
the adjacent frames. This naturally renders the approach robust to occlusion or
motion blur in individual frames. Our framework does not require additional
supervision, as it optimizes sampling locations directly with respect to object
detection performance. Our STSN outperforms the state-of-the-art on the
ImageNet VID dataset and compared to prior video object detection methods it
uses a simpler design, and does not require optical flow data for training.
Gedas Bertasius Lorenzo Torresani Jianbo Shi