As the goal of this paper is not to develop a better human detector or tracker
for building the person tubelets, we use the Faster-RCNN detector released
publicly by Ch eron et al. [6] for all our evaluations on the UCF101-24 validation
set. This detector was originally trained on COCO and then netuned on the
UCF101-24 training set using Detectron [13].
The eect of tubelet sampling For the tubelets of length K= 16 that we use,
there is a mean of 33.1 tubelets per video in the UCF101-24 dataset. In computing
this, we only consider tubelets which have a spatio-temporal IoU of less than 0.5
with each other. More tubelets would be obtained if we counted one from each
frame of the video.
As we can t a maximum of 16 tubelets onto a 16GB Nvidia V100 GPU, it
is clear that it is necessary to sample the tubelets in each bag. Note that UCF
videos often have a high number of tubelets, as there are often many people
in the video who are not labelled as performing an action. As described in the
previous subsection, this is also a signicant source of noise.
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid