title: DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End
    Full-Length Song Generation with Latent Diffusion
summary: Recent advancements in music generation have garnered significant attention,
  yet existing approaches face critical limitations. Some current generative
  models can only synthesize either the vocal track or the accompaniment track.
  While some models can generate combined vocal and accompaniment, they typically
  rely on meticulously designed multi-stage cascading architectures and intricate
  data pipelines, hindering scalability. Additionally, most systems are
  restricted to generating short musical segments rather than full-length songs.
  Furthermore, widely used language model-based methods suffer from slow
  inference speeds. To address these challenges, we propose DiffRhythm, the first
  latent diffusion-based song generation model capable of synthesizing complete
  songs with both vocal and accompaniment for durations of up to 4m45s in only
  ten seconds, maintaining high musicality and intelligibility. Despite its
  remarkable capabilities, DiffRhythm is designed to be simple and elegant: it
  eliminates the need for complex data preparation, employs a straightforward
  model structure, and requires only lyrics and a style prompt during inference.
  Additionally, its non-autoregressive structure ensures fast inference speeds.
  This simplicity guarantees the scalability of DiffRhythm. Moreover, we release
  the complete training code along with the pre-trained model on large-scale data
  to promote reproducibility and further research.
paper_authors._id: 67c6a15e21d722b4248bd9c2
paper_authors.name: Ziqian Ning
paper_authors._id: 67c6a15e21d722b4248bd9c3
paper_authors.name: Huakang Chen
paper_authors._id: 67c6a15e21d722b4248bd9c4
paper_authors.name: Yuepeng Jiang
paper_authors._id: 67c6a15e21d722b4248bd9c5
paper_authors.name: Chunbo Hao
paper_authors._id: 67c6a15e21d722b4248bd9c6
paper_authors.name: Guobin Ma
paper_authors._id: 67c6a15e21d722b4248bd9c7
paper_authors.name: Shuai Wang
paper_authors._id: 67c6a15e21d722b4248bd9c8
paper_authors.name: Jixun Yao
paper_authors._id: 67c6a15e21d722b4248bd9c9
paper_authors.name: Lei Xie
paper_title: DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End
    Full-Length Song Generation with Latent Diffusion
paper_summary: Recent advancements in music generation have garnered significant attention,
  yet existing approaches face critical limitations. Some current generative
  models can only synthesize either the vocal track or the accompaniment track.
  While some models can generate combined vocal and accompaniment, they typically
  rely on meticulously designed multi-stage cascading architectures and intricate
  data pipelines, hindering scalability. Additionally, most systems are
  restricted to generating short musical segments rather than full-length songs.
  Furthermore, widely used language model-based methods suffer from slow
  inference speeds. To address these challenges, we propose DiffRhythm, the first
  latent diffusion-based song generation model capable of synthesizing complete
  songs with both vocal and accompaniment for durations of up to 4m45s in only
  ten seconds, maintaining high musicality and intelligibility. Despite its
  remarkable capabilities, DiffRhythm is designed to be simple and elegant: it
  eliminates the need for complex data preparation, employs a straightforward
  model structure, and requires only lyrics and a style prompt during inference.
  Additionally, its non-autoregressive structure ensures fast inference speeds.
  This simplicity guarantees the scalability of DiffRhythm. Moreover, we release
  the complete training code along with the pre-trained model on large-scale data
  to promote reproducibility and further research.
paper_ai_summary: DiffRhythm, a latent diffusion-based model, synthesizes full-length songs with both vocals and accompaniment efficiently, using simple and scalable methods.
paper_ai_keywords: latent diffusion song generation vocals accompaniment inference speed non-autoregressive structure pre-trained model large-scale data