Multi-Microphone Speech Emotion Recognition using the Hierarchical Token-semantic Audio Transformer Architecture
<|reference_start|>Multi-Microphone Speech Emotion Recognition using the Hierarchical Token-semantic Audio Transformer Architecture: The performance of most emotion recognition systems degrades in real-life situations ('in the wild' scenarios) where the audio is contaminated by reverberation. Our study explores new methods to alleviate the performance degradation of SER algorithms and develop a more robust system for adverse conditions. We propose processing multi-microphone signals to address these challenges and improve emotion classification accuracy. We adopt a state-of-the-art transformer model, the HTS-AT, to handle multi-channel audio inputs. We evaluate two strategies: averaging mel-spectrograms across channels and summing patch-embedded representations. Our multi-microphone model achieves superior performance compared to single-channel baselines when tested on real-world reverberant environments.<|reference_end|>
@article{cohen2024multi-microphone,
    title={Multi-Microphone Speech Emotion Recognition using the Hierarchical
  Token-semantic Audio Transformer Architecture},
    author={Ohad Cohen, Gershon Hazan and Sharon Gannot},
    journal={arXiv preprint arXiv:2406.03272},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.03272},
    primaryClass={eess.AS cs.AI cs.LG}
}