Tab. 5. Note that our weakly-supervised result from sub-clips of 10 seconds
(Tab. 4) outperforms the original fully-supervised baseline using introduced by
the AVA dataset [ 14] that uses both RGB and optical 
ow as inputs. Our model,
on the other hand, only uses RGB as its input modality. Our SlowFast model
14 A. Arnab et al.
Fig. 5. Per-class results, in terms of the Frame AP, on the AVA dataset under dierent
levels of supervision (the longer the sub-clip duration, the weaker the supervision). For
clarity, the rst 30 classes, ranked using the fully-supervised model, are shown. As
expected, action classes benet from stronger supervision, while some classes, such as
\watch", \get up" and \close" are very dicult to learn from long sub-clips.
performs similarly to the published results of the original authors [ 9]. Note that
we have not used Non-local [ 49], test-time augmentation or ensembling which
are all complementary methods to improve performance [ 9]. We can see that
in contrast to the UCF dataset in the previous section, our person detector is
accurate on AVA, and so a Fast-RCNN-style detector using person tubelets as
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid