TSELM: Target Speaker Extraction using Discrete Tokens and Language Models
<|reference_start|>TSELM: Target Speaker Extraction using Discrete Tokens and Language Models: We propose TSELM, a novel target speaker extraction network that leverages discrete tokens and language models. TSELM utilizes multiple discretized layers from WavLM as input tokens and incorporates cross-attention mechanisms to integrate target speaker information. Language models are employed to capture the sequence dependencies, while a scalable HiFi-GAN is used to reconstruct the audio from the tokens. By applying a cross-entropy loss, TSELM models the probability distribution of output tokens, thus converting the complex regression problem of audio generation into a classification task. Experimental results show that TSELM achieves excellent results in speech quality and comparable results in speech intelligibility.<|reference_end|>
@article{tang2024tselm:,
    title={TSELM: Target Speaker Extraction using Discrete Tokens and Language
  Models},
    author={Beilong Tang, Bang Zeng, Ming Li},
    journal={arXiv preprint arXiv:2409.07841},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.07841},
    primaryClass={cs.SD cs.LG eess.AS}
}