Pose-guided multi-task video transformer for driver action recognition
<|reference_start|>Pose-guided multi-task video transformer for driver action recognition: We investigate the task of identifying situations of distracted driving through analysis of in-car videos. To tackle this challenge we introduce a multi-task video transformer that predicts both distracted actions and driver pose. Leveraging VideoMAEv2, a large pre-trained architecture, our approach incorporates semantic information from human keypoint locations to enhance action recognition and decrease computational overhead by minimizing the number of spatio-temporal tokens. By guiding token selection with pose and class information, we notably reduce the model's computational requirements while preserving the baseline accuracy. Our model surpasses existing state-of-the art results in driver action recognition while exhibiting superior efficiency compared to current video transformer-based approaches.<|reference_end|>
@article{pizarro2024pose-guided,
    title={Pose-guided multi-task video transformer for driver action recognition},
    author={Ricardo Pizarro and Roberto Valle and Luis Miguel Bergasa and Jos'e
  M. Buenaposada and Luis Baumela},
    journal={arXiv preprint arXiv:2407.13750},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.13750},
    primaryClass={cs.CV}
}