GMP-TL: Gender-augmented Multi-scale Pseudo-label Enhanced Transfer Learning for Speech Emotion Recognition
<|reference_start|>GMP-TL: Gender-augmented Multi-scale Pseudo-label Enhanced Transfer Learning for Speech Emotion Recognition: The continuous evolution of pre-trained speech models has greatly advanced Speech Emotion Recognition (SER). However, current research typically relies on utterance-level emotion labels, inadequately capturing the complexity of emotions within a single utterance. In this paper, we introduce GMP-TL, a novel SER framework that employs gender-augmented multi-scale pseudo-label (GMP) based transfer learning to mitigate this gap. Specifically, GMP-TL initially uses the pre-trained HuBERT, implementing multi-task learning and multi-scale k-means clustering to acquire frame-level GMPs. Subsequently, to fully leverage frame-level GMPs and utterance-level emotion labels, a two-stage model fine-tuning approach is presented to further optimize GMP-TL. Experiments on IEMOCAP show that our GMP-TL attains a WAR of 80.0% and an UAR of 82.0%, achieving superior performance compared to state-of-the-art unimodal SER methods while also yielding comparable results to multimodal SER approaches.<|reference_end|>
@article{pan2024gmp-tl:,
    title={GMP-TL: Gender-augmented Multi-scale Pseudo-label Enhanced Transfer
  Learning for Speech Emotion Recognition},
    author={Yu Pan, Yuguang Yang, Heng Lu, Lei Ma, Jianjun Zhao},
    journal={arXiv preprint arXiv:2405.02151},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.02151},
    primaryClass={cs.SD cs.AI eess.AS}
}