MultiCounter: Multiple Action Agnostic Repetition Counting in Untrimmed Videos
<|reference_start|>MultiCounter: Multiple Action Agnostic Repetition Counting in Untrimmed Videos: Multi-instance Repetitive Action Counting (MRAC) aims to estimate the number of repetitive actions performed by multiple instances in untrimmed videos, commonly found in human-centric domains like sports and exercise. In this paper, we propose MultiCounter, a fully end-to-end deep learning framework that enables simultaneous detection, tracking, and counting of repetitive actions of multiple human instances. Specifically, MultiCounter incorporates two novel modules: 1) mixed spatiotemporal interaction for efficient context correlation across consecutive frames, and 2) task-specific heads for accurate perception of periodic boundaries and generalization for action-agnostic human instances. We train MultiCounter on a synthetic dataset called MultiRep generated from annotated real-world videos. Experiments on the MultiRep dataset validate the fundamental challenge of MRAC tasks and showcase the superiority of our proposed model. Compared to ByteTrack+RepNet, a solution that combines an advanced tracker with a single repetition counter, MultiCounter substantially improves Period-mAP by 41.0%, reduces AvgMAE by 58.6%, and increases AvgOBO 1.48 times. This sets a new benchmark in the field of MRAC. Moreover, MultiCounter runs in real-time on a commodity GPU server and is insensitive to the number of human instances in a video.<|reference_end|>
@article{tang2024multicounter:,
    title={MultiCounter: Multiple Action Agnostic Repetition Counting in Untrimmed
  Videos},
    author={Yin Tang, Wei Luo, Jinrui Zhang, Wei Huang, Ruihai Jing, and Deyu
  Zhang},
    journal={arXiv preprint arXiv:2409.04035},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.04035},
    primaryClass={cs.CV}
}