Active Learning with Task Adaptation Pre-training for Speech Emotion Recognition
<|reference_start|>Active Learning with Task Adaptation Pre-training for Speech Emotion Recognition: Speech emotion recognition (SER) has garnered increasing attention due to its wide range of applications in various fields, including human-machine interaction, virtual assistants, and mental health assistance. However, existing SER methods often overlook the information gap between the pre-training speech recognition task and the downstream SER task, resulting in sub-optimal performance. Moreover, current methods require much time for fine-tuning on each specific speech dataset, such as IEMOCAP, which limits their effectiveness in real-world scenarios with large-scale noisy data. To address these issues, we propose an active learning (AL)-based fine-tuning framework for SER, called \textsc{After}, that leverages task adaptation pre-training (TAPT) and AL methods to enhance performance and efficiency. Specifically, we first use TAPT to minimize the information gap between the pre-training speech recognition task and the downstream speech emotion recognition task. Then, AL methods are employed to iteratively select a subset of the most informative and diverse samples for fine-tuning, thereby reducing time consumption. Experiments demonstrate that our proposed method \textsc{After}, using only 20\% of samples, improves accuracy by 8.45\% and reduces time consumption by 79\%. The additional extension of \textsc{After} and ablation studies further confirm its effectiveness and applicability to various real-world scenarios. Our source code is available on Github for reproducibility. (https://github.com/Clearloveyuan/AFTER).<|reference_end|>
@article{li2024active,
    title={Active Learning with Task Adaptation Pre-training for Speech Emotion
  Recognition},
    author={Dongyuan Li and Ying Zhang and Yusong Wang and Funakoshi Kataro and
  Manabu Okumura},
    journal={arXiv preprint arXiv:2405.00307},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.00307},
    primaryClass={cs.SD cs.AI eess.AS}
}