Audio xLSTMs: Learning Self-Supervised Audio Representations with xLSTMs
<|reference_start|>Audio xLSTMs: Learning Self-Supervised Audio Representations with xLSTMs: While the transformer has emerged as the eminent neural architecture, several independent lines of research have emerged to address its limitations. Recurrent neural approaches have also observed a lot of renewed interest, including the extended long short-term memory (xLSTM) architecture, which reinvigorates the original LSTM architecture. However, while xLSTMs have shown competitive performance compared to the transformer, their viability for learning self-supervised general-purpose audio representations has not yet been evaluated. This work proposes Audio xLSTM (AxLSTM), an approach to learn audio representations from masked spectrogram patches in a self-supervised setting. Pretrained on the AudioSet dataset, the proposed AxLSTM models outperform comparable self-supervised audio spectrogram transformer (SSAST) baselines by up to 20% in relative performance across a set of ten diverse downstream tasks while having up to 45% fewer parameters.<|reference_end|>
@article{yadav2024audio,
    title={Audio xLSTMs: Learning Self-Supervised Audio Representations with xLSTMs},
    author={Sarthak Yadav, Sergios Theodoridis, Zheng-Hua Tan},
    journal={arXiv preprint arXiv:2408.16568},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.16568},
    primaryClass={cs.SD eess.AS}
}