SER Evals: In-domain and Out-of-domain Benchmarking for Speech Emotion Recognition
<|reference_start|>SER Evals: In-domain and Out-of-domain Benchmarking for Speech Emotion Recognition: Speech emotion recognition (SER) has made significant strides with the advent of powerful self-supervised learning (SSL) models. However, the generalization of these models to diverse languages and emotional expressions remains a challenge. We propose a large-scale benchmark to evaluate the robustness and adaptability of state-of-the-art SER models in both in-domain and out-of-domain settings. Our benchmark includes a diverse set of multilingual datasets, focusing on less commonly used corpora to assess generalization to new data. We employ logit adjustment to account for varying class distributions and establish a single dataset cluster for systematic evaluation. Surprisingly, we find that the Whisper model, primarily designed for automatic speech recognition, outperforms dedicated SSL models in cross-lingual SER. Our results highlight the need for more robust and generalizable SER models, and our benchmark serves as a valuable resource to drive future research in this direction.<|reference_end|>
@article{osman2024ser,
    title={SER Evals: In-domain and Out-of-domain Benchmarking for Speech Emotion
  Recognition},
    author={Mohamed Osman, Daniel Z. Kaplan, Tamer Nadeem},
    journal={arXiv preprint arXiv:2408.07851},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.07851},
    primaryClass={cs.CL cs.AI}
}