respectively, suggesting that bounding-box level annotations are not required for
training action recognition models if the video clips are annotated over short
temporal intervals. Understandably, the results from N= 900, where we use the
video-level annotations over the whole 15-minute clip are the worst as it is the
most dicult setting.
Figure 5 further analyses the per-class results for the dierent levels of
supervision presented in Tab. 4. As expected, stronger levels of supervision
(shorter sub-clip durations) result in better per-class accuracy. However, some
action classes are aected more than others by weaker labels (longer sub-clips).
Examples of this include \sing to" and \listen to" which show a larger dierence
to the fully-supervised baseline than other classes. Moreover, some classes such
as \watch (a person)", \get up", \close (e.g., a door, a box)" and \hand clap"
perform reasonably when trained with sub-clips ( N10), but much more poorly
when trained with longer sub-clips.
Finally, we compare our fully-supervised baseline to the state-of-the-art in
Tab. 5. Note that our weakly-supervised result from sub-clips of 10 seconds
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid