title: MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for
    Zero-Shot Speech Synthesis
summary: While recent zero-shot text-to-speech (TTS) models have significantly
  improved speech quality and expressiveness, mainstream systems still suffer
  from issues related to speech-text alignment modeling: 1) models without
  explicit speech-text alignment modeling exhibit less robustness, especially for
  hard sentences in practical applications; 2) predefined alignment-based models
  suffer from naturalness constraints of forced alignments. This paper introduces
  MegaTTS 3, a TTS system featuring an innovative sparse alignment
  algorithm that guides the latent diffusion transformer (DiT). Specifically, we
  provide sparse alignment boundaries to MegaTTS 3 to reduce the difficulty of
  alignment without limiting the search space, thereby achieving high
  naturalness. Moreover, we employ a multi-condition classifier-free guidance
  strategy for accent intensity adjustment and adopt the piecewise rectified flow
  technique to accelerate the generation process. Experiments demonstrate that
  MegaTTS 3 achieves state-of-the-art zero-shot TTS speech quality and supports
  highly flexible control over accent intensity. Notably, our system can generate
  high-quality one-minute speech with only 8 sampling steps. Audio samples are
  available at https://sditdemo.github.io/sditdemo/.
paper_authors._id: 67ee3cf7042e8ba3e96fc1e0
paper_authors.name: Ziyue Jiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-03T08:28:36.564Z
paper_authors._id: 67dbdf261956dcedf0f0a7e1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/BRzXe_7jytEYadJ0byMyD.png
paper_authors.fullname: ZiyueJiang
paper_authors.type: user
paper_authors.user: ZiyueJiang
paper_authors._id: 67ee3cf7042e8ba3e96fc1e1
paper_authors.name: Yi Ren
paper_authors._id: 67ee3cf7042e8ba3e96fc1e2
paper_authors.name: Ruiqi Li
paper_authors._id: 67ee3cf7042e8ba3e96fc1e3
paper_authors.name: Shengpeng Ji
paper_authors._id: 67ee3cf7042e8ba3e96fc1e4
paper_authors.name: Boyang Zhang
paper_authors._id: 67ee3cf7042e8ba3e96fc1e5
paper_authors.name: Zhenhui Ye
paper_authors._id: 67ee3cf7042e8ba3e96fc1e6
paper_authors.name: Chen Zhang
paper_authors._id: 67ee3cf7042e8ba3e96fc1e7
paper_authors.name: Bai Jionghao
paper_authors._id: 67ee3cf7042e8ba3e96fc1e8
paper_authors.name: Xiaoda Yang
paper_authors._id: 67ee3cf7042e8ba3e96fc1e9
paper_authors.name: Jialong Zuo
paper_authors._id: 67ee3cf7042e8ba3e96fc1ea
paper_authors.name: Yu Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-30T09:56:50.803Z
paper_authors._id: 66569729ea21cfae5f5797c4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66569729ea21cfae5f5797c4/IguwJzljFN3QiEd1bn5BP.jpeg
paper_authors.fullname: Yu Zhang
paper_authors.type: user
paper_authors.user: AaronZ345
paper_authors._id: 67ee3cf7042e8ba3e96fc1eb
paper_authors.name: Rui Liu
paper_authors._id: 67ee3cf7042e8ba3e96fc1ec
paper_authors.name: Xiang Yin
paper_authors._id: 67ee3cf7042e8ba3e96fc1ed
paper_authors.name: Zhou Zhao
paper_title: MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for
    Zero-Shot Speech Synthesis
paper_summary: While recent zero-shot text-to-speech (TTS) models have significantly
  improved speech quality and expressiveness, mainstream systems still suffer
  from issues related to speech-text alignment modeling: 1) models without
  explicit speech-text alignment modeling exhibit less robustness, especially for
  hard sentences in practical applications; 2) predefined alignment-based models
  suffer from naturalness constraints of forced alignments. This paper introduces
  MegaTTS 3, a TTS system featuring an innovative sparse alignment
  algorithm that guides the latent diffusion transformer (DiT). Specifically, we
  provide sparse alignment boundaries to MegaTTS 3 to reduce the difficulty of
  alignment without limiting the search space, thereby achieving high
  naturalness. Moreover, we employ a multi-condition classifier-free guidance
  strategy for accent intensity adjustment and adopt the piecewise rectified flow
  technique to accelerate the generation process. Experiments demonstrate that
  MegaTTS 3 achieves state-of-the-art zero-shot TTS speech quality and supports
  highly flexible control over accent intensity. Notably, our system can generate
  high-quality one-minute speech with only 8 sampling steps. Audio samples are
  available at https://sditdemo.github.io/sditdemo/.
paper_ai_summary: MegaTTS 3, a zero-shot text-to-speech system, uses a sparse alignment algorithm and latent diffusion transformer for high-quality speech generation with flexible accent control.
paper_ai_keywords: sparse alignment algorithm latent diffusion transformer multi-condition classifier-free guidance piecewise rectified flow