We train our network with synchronous stochastic gradient descent (SGD),
using 8 GPUs and a batch size of 4 on each GPU. In our case, each element of a
batch is of a bag from Multiple Instance Learning. Each bag samples a maximum
of 4 tubelets. Each tubelet itself consists of 16 frames.
4 Experiments
4.1 Experimental set-up
We evaluate our method on UCF101-24 and AVA, described in more detail below.
Note that other video datasets such as THUMOS [ 18] and ActivityNet [ 4] are not
suitable for spatiotemporal localisation, as they lack bounding box annotations.
UCF101-24: UCF101-24 is a subset of the UCF101 [ 43] dataset, consisting of 24
action classes with spatio-temporal localisation annotation, released as bounding
box annotations of humans.Although each video contains only a single action
class, it may contain multiple individuals performing the action with dierent
spatial and temporal boundaries. Moreover, there may also be people present in
the video who are not performing any labelled action. Following standard practice,
Uncertainty-Aware Weakly Supervised Action Detection 9
Table 1. Ablation study of dierent variants of our method on the UCF101-24 validation
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid