Flow-TSVAD: Target-Speaker Voice Activity Detection via Latent Flow Matching
<|reference_start|>Flow-TSVAD: Target-Speaker Voice Activity Detection via Latent Flow Matching: Speaker diarization is typically considered a discriminative task, using discriminative approaches to produce fixed diarization results. In this paper, we explore the use of neural network-based generative methods for speaker diarization for the first time. We implement a Flow-Matching (FM) based generative algorithm within the sequence-to-sequence target speaker voice activity detection (Seq2Seq-TSVAD) diarization system. Our experiments reveal that applying the generative method directly to the original binary label sequence space of the TS-VAD output is ineffective. To address this issue, we propose mapping the binary label sequence into a dense latent space before applying the generative algorithm and our proposed Flow-TSVAD method outperforms the Seq2Seq-TSVAD system. Additionally, we observe that the FM algorithm converges rapidly during the inference stage, requiring only two inference steps to achieve promising results. As a generative model, Flow-TSVAD allows for sampling different diarization results by running the model multiple times. Moreover, ensembling results from various sampling instances further enhances diarization performance.<|reference_end|>
@article{chen2024flow-tsvad:,
    title={Flow-TSVAD: Target-Speaker Voice Activity Detection via Latent Flow
  Matching},
    author={Zhengyang Chen, Bing Han, Shuai Wang, Yidi Jiang, Yanmin Qian},
    journal={arXiv preprint arXiv:2409.04859},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.04859},
    primaryClass={cs.SD eess.AS}
}