Uncertainty-Aware Weakly Supervised Action Detection 9
Table 1. Ablation study of dierent variants of our method on the UCF101-24 validation
set. We report the Video mAP at IoU thresholds of 0.2 and 0.5 respectively.
Video AP
0.2 0.5
Weakly supervised baseline 54.3 29.7
MIL - LSE pooling 60.1 33.1
MIL - mean pooling 60.3 33.0
MIL - max pooling 60.7 33.5
MIL - max pooling, uncertainty 61.7 35.0
Fully supervised 69.3 43.6
we use the corrected annotations of [ 41] and report the mean average precision
at a video level (Video AP) for the rst split of the dataset. For evaluating the
Video AP, we link tubelets together using the algorithm of [19].
AVA [ 14]:This dataset consists of 430, 15-minute video clips obtained from
movies. 80 atomic visual actions are annotated exhaustively for all people in
the video, where one person is often simultaneously performing multiple actions.
The dataset annotates keyframes at every second in the video. Following standard practice, we report the Frame AP at an IoU threshold of 0.5 using v2.2
annotations.
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid