in the video who are not labelled as performing an action. As described in the
previous subsection, this is also a signicant source of noise.
Table 2 shows the eect of changing the batch size (number of bags), and the
number of tubelets sampled per bag, such that GPU memory usage is maximised.
We can see that the uncertainty loss helps in all cases and that accuracy decreases
with low batch sizes. We believe this is due to batch normalisation statistics
being too correlated when more tubes are from the same video.
Comparison to state-of-the-art Table 3 compares our results to the state-of-the-art.
The bottom-half of the table shows that we outperform previous weakly-supervised
Uncertainty-Aware Weakly Supervised Action Detection 11
Table 2. The eect of the number of bags in each training batch on accuracy (Video
AP at 0.5). The uncertainty loss improves accuracy in all scenarios. Although fewer, but
larger, bags can reduce the noise due to sampling, they also cause batch normalisation
statistics to be too correlated, reducing accuracy.
Number of bags in batch Tubelets sampled per bag Video AP without uncertainty Video AP with uncertainty
4 4 33.5 35.0
3 5 33.6 34.1
2 8 33.3 34.2
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid