the precision of our person tubelets on the training set is only 21.1%. A major
contributing factor to this is that UCF action annotations are not exhaustive
{ there may be people in the video who are not labelled at all as they are not
performing an annotated action. These people will, however, still be detected by
a COCO-trained detector and considered as false positives during this evaluation.
The fact that we are able to train our model with these annotations demonstrates the ability of our multiple instance learning method to handle label noise
in the training set. The inconsistencies in the UCF101-24 dataset labelling are
detailed further in the supplementary, and has also been noted previously by
Ch eron et al. [6].
Noise in the person detections are not a problem for the training of our
fully-supervised baseline, as it is trained with ground-truth boxes in addition
to predicted boxes. As we have box-level supervision in this case, predicted
detections which have an IoU of more than 0.5 with a ground-truth detection
are assigned the label of the ground-truth box, or the negative label otherwise,
during fully-supervised training.
As the goal of this paper is not to develop a better human detector or tracker
for building the person tubelets, we use the Faster-RCNN detector released
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid