for this: Firstly, due to computational constraints, we cannot process a whole bag
at a time, but must instead sample instances from a bag. It is therefore possible
to sample a bag that does not contain any tubelets with the labelled action. The
Uncertainty-Aware Weakly Supervised Action Detection 7
Predicted 
uncertainty
, 


Predicted probability
, 


b
a
c
Wrong prediction. Low 
uncertainty. Highest loss
Correct prediction. Low 
uncertainty. Lowest loss
Incorrect prediction. High uncertainty. Moderate loss
Fig. 3. The loss surface of our uncertainty-based loss (Eq. 6). The ground truth binary
label in this example is 1. Hence, the loss is minimised when the network predicts
a high probability and a low uncertainty (point \a"). However, making an incorrect
prediction with a high uncertainty is not penalised as much (point \b"), and is suitable
for cases when the input bags are noisy and the bag-level label is not present in any of
the tubelets. Finally, predicting the incorrect label with a low uncertainty is penalised
the most (point \c"). Best viewed in colour.
likelihood of this occurring is inversely proportional to the ratio of the duration
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid