SAML: Speaker Adaptive Mixture of LoRA Experts for End-to-End ASR
<|reference_start|>SAML: Speaker Adaptive Mixture of LoRA Experts for End-to-End ASR: Mixture-of-experts (MoE) models have achieved excellent results in many tasks. However, conventional MoE models are often very large, making them challenging to deploy on resource-constrained edge devices. In this paper, we propose a novel speaker adaptive mixture of LoRA experts (SAML) approach, which uses low-rank adaptation (LoRA) modules as experts to reduce the number of trainable parameters in MoE. Specifically, SAML is applied to the quantised and personalised end-to-end automatic speech recognition models, which combines test-time speaker adaptation to improve the performance of heavily compressed models in speaker-specific scenarios. Experiments have been performed on the LibriSpeech and the TED-LIUM 3 corpora. Remarkably, with a 7x reduction in model size, 29.1% and 31.1% relative word error rate reductions were achieved on the quantised Whisper model and Conformer-based attention-based encoder-decoder ASR model respectively, comparing to the original full precision models.<|reference_end|>
@article{zhao2024saml:,
    title={SAML: Speaker Adaptive Mixture of LoRA Experts for End-to-End ASR},
    author={Qiuming Zhao, Guangzhi Sun, Chao Zhang, Mingxing Xu, Thomas Fang Zheng},
    journal={arXiv preprint arXiv:2406.19706},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.19706},
    primaryClass={cs.SD eess.AS}
}