title: SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song
    Generation
summary: Text-to-song generation, the task of creating vocals and accompaniment from
  textual inputs, poses significant challenges due to domain complexity and data
  scarcity. Existing approaches often employ multi-stage generation procedures,
  resulting in cumbersome training and inference pipelines. In this paper, we
  propose SongGen, a fully open-source, single-stage auto-regressive transformer
  designed for controllable song generation. The proposed model facilitates
  fine-grained control over diverse musical attributes, including lyrics and
  textual descriptions of instrumentation, genre, mood, and timbre, while also
  offering an optional three-second reference clip for voice cloning. Within a
  unified auto-regressive framework, SongGen supports two output modes: mixed
  mode, which generates a mixture of vocals and accompaniment directly, and
  dual-track mode, which synthesizes them separately for greater flexibility in
  downstream applications. We explore diverse token pattern strategies for each
  mode, leading to notable improvements and valuable insights. Furthermore, we
  design an automated data preprocessing pipeline with effective quality control.
  To foster community engagement and future research, we will release our model
  weights, training code, annotated data, and preprocessing pipeline. The
  generated samples are showcased on our project page at
  https://liuzh-19.github.io/SongGen/ , and the code will be available at
  https://github.com/LiuZH-19/SongGen .
paper_authors._id: 67b6c696e9b901edeaf320d5
paper_authors.name: Zihan Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-20T16:07:49.211Z
paper_authors._id: 65f33b1c9f7970ccc0234cbf
paper_authors.avatarUrl: /avatars/99fbab303912e3674663251c04279907.svg
paper_authors.fullname: Zihan Liu
paper_authors.type: user
paper_authors.user: zihanliu
paper_authors._id: 67b6c696e9b901edeaf320d6
paper_authors.name: Shuangrui Ding
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-20T16:07:35.659Z
paper_authors._id: 65a7c0335e79abfa2ec30c52
paper_authors.avatarUrl: /avatars/2f62f83f9c5c4cc9444571f067cd85b7.svg
paper_authors.fullname: Shuangrui Ding
paper_authors.type: user
paper_authors.user: Mar2Ding
paper_authors._id: 67b6c696e9b901edeaf320d7
paper_authors.name: Zhixiong Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-20T16:07:29.113Z
paper_authors._id: 64b75f59324038715942628b
paper_authors.avatarUrl: /avatars/809c533138c2a69acbc3e42a143bbf34.svg
paper_authors.fullname: Zhixiong Zhang
paper_authors.type: user
paper_authors.user: zzxlmlzcg2
paper_authors._id: 67b6c696e9b901edeaf320d8
paper_authors.name: Xiaoyi Dong
paper_authors._id: 67b6c696e9b901edeaf320d9
paper_authors.name: Pan Zhang
paper_authors._id: 67b6c696e9b901edeaf320da
paper_authors.name: Yuhang Zang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-20T16:07:11.149Z
paper_authors._id: 63859cf3b2906edaf83af9f0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/mCgynEtoXdILvzyoPexii.png
paper_authors.fullname: Yuhang Zang
paper_authors.type: user
paper_authors.user: yuhangzang
paper_authors._id: 67b6c696e9b901edeaf320db
paper_authors.name: Yuhang Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-20T16:07:04.833Z
paper_authors._id: 65000bef18830fabea469fdd
paper_authors.avatarUrl: /avatars/b320c77dfad039d9f9c54127f610d44f.svg
paper_authors.fullname: Cao Yuhang
paper_authors.type: user
paper_authors.user: yhcao
paper_authors._id: 67b6c696e9b901edeaf320dc
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-20T16:06:53.745Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 67b6c696e9b901edeaf320dd
paper_authors.name: Jiaqi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-20T16:06:47.581Z
paper_authors._id: 64b4eec4faa3181a5eab9c46
paper_authors.avatarUrl: /avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
paper_authors.fullname: Jiaqi Wang
paper_authors.type: user
paper_authors.user: myownskyW7
paper_title: SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song
    Generation
paper_summary: Text-to-song generation, the task of creating vocals and accompaniment from
  textual inputs, poses significant challenges due to domain complexity and data
  scarcity. Existing approaches often employ multi-stage generation procedures,
  resulting in cumbersome training and inference pipelines. In this paper, we
  propose SongGen, a fully open-source, single-stage auto-regressive transformer
  designed for controllable song generation. The proposed model facilitates
  fine-grained control over diverse musical attributes, including lyrics and
  textual descriptions of instrumentation, genre, mood, and timbre, while also
  offering an optional three-second reference clip for voice cloning. Within a
  unified auto-regressive framework, SongGen supports two output modes: mixed
  mode, which generates a mixture of vocals and accompaniment directly, and
  dual-track mode, which synthesizes them separately for greater flexibility in
  downstream applications. We explore diverse token pattern strategies for each
  mode, leading to notable improvements and valuable insights. Furthermore, we
  design an automated data preprocessing pipeline with effective quality control.
  To foster community engagement and future research, we will release our model
  weights, training code, annotated data, and preprocessing pipeline. The
  generated samples are showcased on our project page at
  https://liuzh-19.github.io/SongGen/ , and the code will be available at
  https://github.com/LiuZH-19/SongGen .
paper_ai_summary: SongGen is a unified, single-stage, auto-regressive transformer model for text-to-song generation, offering controllable attributes and flexible output modes.
paper_ai_keywords: auto-regressive transformer multi-stage generation voice cloning mixed mode dual-track mode token pattern strategies automated data preprocessing pipeline