the most (point \c"). Best viewed in colour.
likelihood of this occurring is inversely proportional to the ratio of the duration
of the labelled action to the total video length. Secondly, in a weakly-supervised
scenario, we use person detectors that are not trained on the video dataset of
interest. Consequently, there can be failures in the detector, especially when
there is a large domain gap between the detector's training distribution and the
video dataset. False negatives (missing detections for people in the scene) are a
particular issue because it is possible that we do not have a single person tubelet
in the bag that corresponds to the labelled action.
Therefore, there are cases when there is no tubelet which actually has the baglevel label. To handle these cases, inspired by [ 22,32], we modify the network to
additionally predict the uncertainty 2RCfor each binary label for all tubelets
in the bag. Intuitively, to minimise the training error, the network can predict the
bag-level label with low uncertainty or it can predict a high uncertainty to avoid
being penalised heavily for noisy bags where the bag-level label is not present in
any of the tubelets. The nal loss, in conjunction with the original cross entropy,
is dened as:
L(x;y; ) =1
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid