title: CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency
    Model
summary: Denoising diffusion probabilistic models (DDPMs) have shown promising
  performance for speech synthesis. However, a large number of iterative steps
  are required to achieve high sample quality, which restricts the inference
  speed. Maintaining sample quality while increasing sampling speed has become a
  challenging task. In this paper, we propose a "Co"nsistency "Mo"del-based
  "Speech" synthesis method, CoMoSpeech, which achieve speech synthesis through a
  single diffusion sampling step while achieving high audio quality. The
  consistency constraint is applied to distill a consistency model from a
  well-designed diffusion-based teacher model, which ultimately yields superior
  performances in the distilled CoMoSpeech. Our experiments show that by
  generating audio recordings by a single sampling step, the CoMoSpeech achieves
  an inference speed more than 150 times faster than real-time on a single NVIDIA
  A100 GPU, which is comparable to FastSpeech2, making diffusion-sampling based
  speech synthesis truly practical. Meanwhile, objective and subjective
  evaluations on text-to-speech and singing voice synthesis show that the
  proposed teacher models yield the best audio quality, and the one-step sampling
  based CoMoSpeech achieves the best inference speed with better or comparable
  audio quality to other conventional multi-step diffusion model baselines. Audio
  samples are available at https://comospeech.github.io/.
paper_authors._id: 645d9731f4f49de580a07751
paper_authors.name: Zhen Ye
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-06T15:59:15.666Z
paper_authors._id: 645f172d7c6bff8577353d1a
paper_authors.avatarUrl: /avatars/a83682e1343809257b082b78d58c582a.svg
paper_authors.fullname: ZhenYE
paper_authors.type: user
paper_authors.user: ZhenYe234
paper_authors._id: 645d9731f4f49de580a07752
paper_authors.name: Wei Xue
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-24T11:14:28.337Z
paper_authors._id: 6628adb14277eae0da5eee28
paper_authors.avatarUrl: /avatars/6cb41b80cc5e014e455dfc2a22682e64.svg
paper_authors.fullname: HKUST Audio
paper_authors.type: user
paper_authors.user: HKUST-Audio
paper_authors._id: 645d9731f4f49de580a07753
paper_authors.name: Xu Tan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-05T08:39:45.256Z
paper_authors._id: 5f1040b6e9d71719e3be71d2
paper_authors.avatarUrl: /avatars/a2f28940236ae625ed3810ad62e343ff.svg
paper_authors.fullname: Xu Tan
paper_authors.type: user
paper_authors.user: xutan
paper_authors._id: 645d9731f4f49de580a07754
paper_authors.name: Jie Chen
paper_authors._id: 645d9731f4f49de580a07755
paper_authors.name: Qifeng Liu
paper_authors._id: 645d9731f4f49de580a07756
paper_authors.name: Yike Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-02T14:56:05.843Z
paper_authors._id: 64762b325f70f9b2d0ade28e
paper_authors.avatarUrl: /avatars/1f37382724b475ece805f943a8858acd.svg
paper_authors.fullname: Yike Guo
paper_authors.type: user
paper_authors.user: SuaLily
paper_title: CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency
    Model
paper_summary: Denoising diffusion probabilistic models (DDPMs) have shown promising
  performance for speech synthesis. However, a large number of iterative steps
  are required to achieve high sample quality, which restricts the inference
  speed. Maintaining sample quality while increasing sampling speed has become a
  challenging task. In this paper, we propose a "Co"nsistency "Mo"del-based
  "Speech" synthesis method, CoMoSpeech, which achieve speech synthesis through a
  single diffusion sampling step while achieving high audio quality. The
  consistency constraint is applied to distill a consistency model from a
  well-designed diffusion-based teacher model, which ultimately yields superior
  performances in the distilled CoMoSpeech. Our experiments show that by
  generating audio recordings by a single sampling step, the CoMoSpeech achieves
  an inference speed more than 150 times faster than real-time on a single NVIDIA
  A100 GPU, which is comparable to FastSpeech2, making diffusion-sampling based
  speech synthesis truly practical. Meanwhile, objective and subjective
  evaluations on text-to-speech and singing voice synthesis show that the
  proposed teacher models yield the best audio quality, and the one-step sampling
  based CoMoSpeech achieves the best inference speed with better or comparable
  audio quality to other conventional multi-step diffusion model baselines. Audio
  samples are available at https://comospeech.github.io/.
paper_ai_keywords: Denoising diffusion probabilistic models (DDPMs) CoMoSpeech diffusion sampling consistency constraint consistency model diffusion-based teacher model text-to-speech singing voice synthesis