in contrast to the UCF dataset in the previous section, our person detector is
accurate on AVA, and so a Fast-RCNN-style detector using person tubelets as
proposals can achieve state-of-the-art results.
5 Conclusion and Future Work
We have proposed a weakly supervised spatio-temporal action detection method
based on Multiple Instance Learning (MIL). Our approach incorporates uncertainty predictions made by the network such that it can better handle noise
in our bags and violations of the standard MIL assumption by predicting a
high uncertainty for noisy bags which cannot be classied correctly. We achieve
state-of-the-art results among weakly supervised methods on the UCF101-24
dataset, and also report the rst weakly-supervised results on AVA, which is the
only large-scale action recognition dataset. Our analysis of the accuracy trade-os
as the time intervals for which sub-clips of the video are annotated will also aid
future dataset annotation eorts.
Future work is to incorporate additional sources of noisy, weakly-labelled
data, such as data which can be scraped o internet search engines.
Uncertainty-Aware Weakly Supervised Action Detection 15
References
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid