SoloSpeech: Enhancing Intelligibility and Quality in Target Speech
  Extraction through a Cascaded Generative Pipeline
Target Speech Extraction (TSE) aims to isolate a target speaker's voice from
a mixture of multiple speakers by leveraging speaker-specific cues, typically
provided as auxiliary audio (a.k.a. cue audio). Although recent advancements in
TSE have primarily employed discriminative models that offer high perceptual
quality, these models often introduce unwanted artifacts, reduce naturalness,
and are sensitive to discrepancies between training and testing environments.
On the other hand, generative models for TSE lag in perceptual quality and
intelligibility. To address these challenges, we present SoloSpeech, a novel
cascaded generative pipeline that integrates compression, extraction,
reconstruction, and correction processes. SoloSpeech features a
speaker-embedding-free target extractor that utilizes conditional information
from the cue audio's latent space, aligning it with the mixture audio's latent
space to prevent mismatches. Evaluated on the widely-used Libri2Mix dataset,
SoloSpeech achieves the new state-of-the-art intelligibility and quality in
target speech extraction and speech separation tasks while demonstrating
exceptional generalization on out-of-domain data and real-world scenarios.
683675205b96c192536256d1
Helin Wang
claimed_verified
2025-05-28T12:12:13.955Z
63ecfb5ec5b3c734085db9ed
/avatars/0b1d03dcd7997ad1daa764fb76f88993.svg
Helin Wang
user
westbrook
683675205b96c192536256d2
Jiarui Hai
claimed_verified
2025-06-06T07:42:22.785Z
6363e9e4969bdae89e10b5ac
https://cdn-avatars.huggingface.co/v1/production/uploads/6363e9e4969bdae89e10b5ac/uR0jz8pBJIQPmKC90OJgI.jpeg
Jiarui Hai
user
Higobeatz
683675205b96c192536256d3
Dongchao Yang
683675205b96c192536256d4
Chen Chen
683675205b96c192536256d5
Kai Li
683675205b96c192536256d6
Junyi Peng
683675205b96c192536256d7
Thomas Thebaud
683675205b96c192536256d8
Laureano Moro Velazquez
683675205b96c192536256d9
Jesus Villalba
683675205b96c192536256da
Najim Dehak
SoloSpeech: Enhancing Intelligibility and Quality in Target Speech
  Extraction through a Cascaded Generative Pipeline
Target Speech Extraction (TSE) aims to isolate a target speaker's voice from
a mixture of multiple speakers by leveraging speaker-specific cues, typically
provided as auxiliary audio (a.k.a. cue audio). Although recent advancements in
TSE have primarily employed discriminative models that offer high perceptual
quality, these models often introduce unwanted artifacts, reduce naturalness,
and are sensitive to discrepancies between training and testing environments.
On the other hand, generative models for TSE lag in perceptual quality and
intelligibility. To address these challenges, we present SoloSpeech, a novel
cascaded generative pipeline that integrates compression, extraction,
reconstruction, and correction processes. SoloSpeech features a
speaker-embedding-free target extractor that utilizes conditional information
from the cue audio's latent space, aligning it with the mixture audio's latent
space to prevent mismatches. Evaluated on the widely-used Libri2Mix dataset,
SoloSpeech achieves the new state-of-the-art intelligibility and quality in
target speech extraction and speech separation tasks while demonstrating
exceptional generalization on out-of-domain data and real-world scenarios.
SoloSpeech, a cascaded generative pipeline, improves target speech extraction and speech separation by addressing artifact introduction, naturalness reduction, and environment mismatches, achieving state-of-the-art intelligibility and quality.
target speech extraction discriminative models generative models speaker-embedding-free target extractor latent space Libri2Mix dataset speech separation