A1
r
(4)
g(fpjg) =1
rlog0
@1
jjjX
jerpj1
A (5)
Max-pooling only considers the top-scoring instance in the bag, and thus naturally
captures the MIL assumption that at least one instance in the bag has the
specied, bag-level label. Moreover, it can also be more robust to instances in
the bag that do not have the bag-level label. However, mean and LSE pooling
have been employed in applications such as weakly-supervised segmentation
[36], object recognition [ 45] and medical imaging [ 24] where multiple instances
in the bag do typically have the bag-level label. Note that higher values of
therhyperparameter for both these functions increases their \peakiness" and
approximates the maximum value. For our scenario, detailed in the next section,
we found max-pooling to be the most appropriate.
3.2 Weakly-supervised spatio-temporal action recognition as
multiple instance learning
Our goal is to learn a model to recognise and localise actions in space and
time given only video-level annotations. To facilitate this, we leverage a person
detector that has been trained on a large image dataset, i.e.Microsoft COCO [ 26].
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid