The dataset annotates keyframes at every second in the video. Following standard practice, we report the Frame AP at an IoU threshold of 0.5 using v2.2
annotations.
4.2 Experiments on UCF101-24
We rst conduct ablation studies of our model on the UCF101-24 dataset. We
discard the spatio-temporal annotations for the whole untrimmed video, and so
our bag in multiple instance learning contains tubelets from the whole video.
Ablation study Table 1 ablates dierent variants of our method: The most na ve
baseline is to not perform any multiple instance learning, and to simply train in
a fully-supervised fashion assuming that the label of a tubelet is the video-level
label. As shown in the rst row of Tab. 1, this method performs the worst as
the assumed tubelet-level labels are often incorrect. The use of multiple instance
learning improves results, with the various aggregation functions performing
similarly. Max-pooling, however, performs the best, and we believe this is because
the max operation is the most suitable for dealing with the noise present in our
tubelets as described in Sec. 3.3. Note that for mean and LSE-pooling, we set
r= 1. Finally, introducing our uncertainty-based loss function improves results
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid