detections. Temporal localisation, on the other hand, is performed by linking
person tubelets through the video as commonly done in the literature [ 19,41,52,6],
since this method can scale to arbitrarily long videos. We use the same algorithm
as Kalogeiton et al. [19] which links together detections within a small temporal
window greedily based on the spatial intersection over union (IoU) between
bounding boxes on consecutive frames.
Finally, note that for a video consisting of Tframes, the bag could consist of
TK+1 person tubelets if a person is detected on each frame of the video, and a
tubelet is started from each frame. Due to memory limitations, it is infeasible to t
an entire bag onto a GPU for training. As a result, we uniformly sample instances
from each bag during training, whilst still retaining the original bag-level label.
This introduces additional noise into the problem, as detailed next.
3.3 Label noise and violation of the standard MIL assumption
The standard MIL assumption, that at least one instance in the bag is assigned
the bag-level label is often violated in our scenario. There are two primary factors
for this: Firstly, due to computational constraints, we cannot process a whole bag
at a time, but must instead sample instances from a bag. It is therefore possible
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid