Domain-Guided Masked Autoencoders for Unique Player Identification
<|reference_start|>Domain-Guided Masked Autoencoders for Unique Player Identification: Unique player identification is a fundamental module in vision-driven sports analytics. Identifying players from broadcast videos can aid with various downstream tasks such as player assessment, in-game analysis, and broadcast production. However, automatic detection of jersey numbers using deep features is challenging primarily due to: a) motion blur, b) low resolution video feed, and c) occlusions. With their recent success in various vision tasks, masked autoencoders (MAEs) have emerged as a superior alternative to conventional feature extractors. However, most MAEs simply zero-out image patches either randomly or focus on where to mask rather than how to mask. Motivated by human vision, we devise a novel domain-guided masking policy for MAEs termed d-MAE to facilitate robust feature extraction in the presence of motion blur for player identification. We further introduce a new spatio-temporal network leveraging our novel d-MAE for unique player identification. We conduct experiments on three large-scale sports datasets, including a curated baseball dataset, the SoccerNet dataset, and an in-house ice hockey dataset. We preprocess the datasets using an upgraded keyframe identification (KfID) module by focusing on frames containing jersey numbers. Additionally, we propose a keyframe-fusion technique to augment keyframes, preserving spatial and temporal context. Our spatio-temporal network showcases significant improvements, surpassing the current state-of-the-art by 8.58%, 4.29%, and 1.20% in the test set accuracies, respectively. Rigorous ablations highlight the effectiveness of our domain-guided masking approach and the refined KfID module, resulting in performance enhancements of 1.48% and 1.84% respectively, compared to original architectures.<|reference_end|>
@article{balaji2024domain-guided,
    title={Domain-Guided Masked Autoencoders for Unique Player Identification},
    author={Bavesh Balaji, Jerrin Bright, Sirisha Rambhatla, Yuhao Chen, Alexander
  Wong, John Zelek and David A Clausi},
    journal={arXiv preprint arXiv:2403.11328},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.11328},
    primaryClass={cs.CV cs.AI}
}