Ours 61.7 35.0
methods by a large margin. The top-half shows that our fully-supervised baseline
is also competitive with the fully-supervised state-of-the-art, although that is not
the main goal of this work. The fully-supervised methods which outperform our
method are based on action detectors which directly predict the person proposals
with the network, and are thus able to handle the person annotation peculiarities
of the UCF101-24 dataset more eectively. We do not observe any issues with
person detections for our experiments on AVA in the next section.
Qualitative Results Figure 4 presents qualitative results of our method. The
rst two rows show success cases of our method where the tubelet detection
and linking have performed well. The third row shows a failure case, since the
basketball player represented by the green track is not actually performing the
\Basketball Dunk" action. According to the UCF101-24 annotations, only the
player represented with the blue track is performing this action. This video clip
is thus an example of a video where there are many people not performing the
action annotated for the video, and is especially challenging for our weaklysupervised method. The fourth row shows a dierent failure case as an error by
12 A. Arnab et al.
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid