META-CAT: Speaker-Informed Speech Embeddings via Meta Information Concatenation for Multi-talker ASR
<|reference_start|>META-CAT: Speaker-Informed Speech Embeddings via Meta Information Concatenation for Multi-talker ASR: We propose a novel end-to-end multi-talker automatic speech recognition (ASR) framework that enables both multi-speaker (MS) ASR and target-speaker (TS) ASR. Our proposed model is trained in a fully end-to-end manner, incorporating speaker supervision from a pre-trained speaker diarization module. We introduce an intuitive yet effective method for masking ASR encoder activations using output from the speaker supervision module, a technique we term Meta-Cat (meta-information concatenation), that can be applied to both MS-ASR and TS-ASR. Our results demonstrate that the proposed architecture achieves competitive performance in both MS-ASR and TS-ASR tasks, without the need for traditional methods, such as neural mask estimation or masking at the audio or feature level. Furthermore, we demonstrate a glimpse of a unified dual-task model which can efficiently handle both MS-ASR and TS-ASR tasks. Thus, this work illustrates that a robust end-to-end multi-talker ASR framework can be implemented with a streamlined architecture, obviating the need for the complex speaker filtering mechanisms employed in previous studies.<|reference_end|>
@article{wang2024meta-cat:,
    title={META-CAT: Speaker-Informed Speech Embeddings via Meta Information
  Concatenation for Multi-talker ASR},
    author={Jinhan Wang, Weiqing Wang, Kunal Dhawan, Taejin Park, Myungjong Kim,
  Ivan Medennikov, He Huang, Nithin Koluguri, Jagadeesh Balam, Boris Ginsburg},
    journal={arXiv preprint arXiv:2409.12352},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.12352},
    primaryClass={eess.AS cs.SD}
}