clip-level annotation. This eectively means that we are obtaining clip-level
annotations for sub-clips of Nseconds from the original AVA video. The weaklysupervised problem gets more dicult as Nincreases, as the sub-clips get longer
and the number of observed labels within each sub-clip increases. Note that when
N= 1, only the spatial localisation ability of the model is being tested, as during
training, it is unknown which of the subclip-level labels correspond to each person
tubelet in the MIL bag. When N > 1, the subclip-level labels can correspond
Uncertainty-Aware Weakly Supervised Action Detection 13
Table 4. Results of our method on the AVA dataset in terms of the Frame mAP at
an IoU threshold of 0.5. We vary the length of the sub-clips from which we extract
clip-level annotations to control the diculty of the weakly supervised problems. FS
denotes a fully-supervised baseline representing the upper bound on performance. A
sub-clip of 900 seconds is an entire AVA video clip.
Sub-clip duration (seconds)
FS 1 5 10 30 60 900
Frame AP 24.9 22.4 18.0 15.8 11.4 9.1 4.2
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid