any of the tubelets. The nal loss, in conjunction with the original cross entropy,
is dened as:
L(x;y; ) =1
2Lce(x;y) + log2(6)
As shown by [ 23], this corresponds to assuming a Boltzmann distribution on the
output of the network with a temperature of 2, and approximately minimising
its log-likelihood.
8 A. Arnab et al.
The loss surface of this probabilistic loss is visualised in Fig. 3. Note how the
loss is the lowest when the predicted label is correct and there is low uncertainty.
However, the loss is not excessive if the incorrect label is predicted with a
high uncertainty. This is in contrast with the standard cross-entropy loss which
penalises incorrect predictions heavily.
3.4 Network architecture and implementation
Our action detector is similar to Fast-RCNN [ 12] using the SlowFast [ 9] video
network architecture based on the ResNet-50 backbone [ 16] pretrained on Kinetics [ 20]. As described in Sec. 3.2, we use region proposals obtained from a
Faster-RCNN detection model trained with Detectron [ 13]. Region-of-interest
features [ 12] are extracted from the last feature map of \res5" using RoIAlign [ 15].
Despite the recent advances in video classification, progress in
spatio-temporal action recognition has lagged behind. A major contributing
factor has been the prohibitive cost of annotating videos frame-by-frame. In
this paper, we present a spatio-temporal action recognition model that is
trained with only video-level labels, which are significantly easier to
annotate. Our method leverages per-frame person detectors which have been
trained on large image datasets within a Multiple Instance Learning framework.
We show how we can apply our method in cases where the standard Multiple
Instance Learning assumption, that each bag contains at least one instance with
the specified label, is invalid using a novel probabilistic variant of MIL
where we estimate the uncertainty of each prediction. Furthermore, we report
the first weakly-supervised results on the AVA dataset and state-of-the-art
results among weakly-supervised methods on UCF101-24.
Anurag Arnab Chen Sun Arsha Nagrani Cordelia Schmid