Action Selection Learning for Multi-label Multi-view Action Recognition
<|reference_start|>Action Selection Learning for Multi-label Multi-view Action Recognition: Multi-label multi-view action recognition aims to recognize multiple concurrent or sequential actions from untrimmed videos captured by multiple cameras. Existing work has focused on multi-view action recognition in a narrow area with strong labels available, where the onset and offset of each action are labeled at the frame-level. This study focuses on real-world scenarios where cameras are distributed to capture a wide-range area with only weak labels available at the video-level. We propose the method named MultiASL (Multi-view Action Selection Learning), which leverages action selection learning to enhance view fusion by selecting the most useful information from different viewpoints. The proposed method includes a Multi-view Spatial-Temporal Transformer video encoder to extract spatial and temporal features from multi-viewpoint videos. Action Selection Learning is employed at the frame-level, using pseudo ground-truth obtained from weak labels at the video-level, to identify the most relevant frames for action recognition. Experiments in a real-world office environment using the MM-Office dataset demonstrate the superior performance of the proposed method compared to existing methods.<|reference_end|>
@article{nguyen2024action,
    title={Action Selection Learning for Multi-label Multi-view Action Recognition},
    author={Trung Thanh Nguyen, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide},
    journal={arXiv preprint arXiv:2410.03302},
    year={2024},
    doi={10.1145/3696409.3700211},
    archivePrefix={arXiv},
    eprint={2410.03302},
    primaryClass={cs.CV}
}