SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song
  Generation
Text-to-song generation, the task of creating vocals and accompaniment from
textual inputs, poses significant challenges due to domain complexity and data
scarcity. Existing approaches often employ multi-stage generation procedures,
resulting in cumbersome training and inference pipelines. In this paper, we
propose SongGen, a fully open-source, single-stage auto-regressive transformer
designed for controllable song generation. The proposed model facilitates
fine-grained control over diverse musical attributes, including lyrics and
textual descriptions of instrumentation, genre, mood, and timbre, while also
offering an optional three-second reference clip for voice cloning. Within a
unified auto-regressive framework, SongGen supports two output modes: mixed
mode, which generates a mixture of vocals and accompaniment directly, and
dual-track mode, which synthesizes them separately for greater flexibility in
downstream applications. We explore diverse token pattern strategies for each
mode, leading to notable improvements and valuable insights. Furthermore, we
design an automated data preprocessing pipeline with effective quality control.
To foster community engagement and future research, we will release our model
weights, training code, annotated data, and preprocessing pipeline. The
generated samples are showcased on our project page at
https://liuzh-19.github.io/SongGen/ , and the code will be available at
https://github.com/LiuZH-19/SongGen .
67b6c696e9b901edeaf320d5
Zihan Liu
admin_assigned
2025-02-20T16:07:49.211Z
65f33b1c9f7970ccc0234cbf
/avatars/99fbab303912e3674663251c04279907.svg
Zihan Liu
user
zihanliu
67b6c696e9b901edeaf320d6
Shuangrui Ding
admin_assigned
2025-02-20T16:07:35.659Z
65a7c0335e79abfa2ec30c52
/avatars/2f62f83f9c5c4cc9444571f067cd85b7.svg
Shuangrui Ding
user
Mar2Ding
67b6c696e9b901edeaf320d7
Zhixiong Zhang
admin_assigned
2025-02-20T16:07:29.113Z
64b75f59324038715942628b
/avatars/809c533138c2a69acbc3e42a143bbf34.svg
Zhixiong Zhang
user
zzxlmlzcg2
67b6c696e9b901edeaf320d8
Xiaoyi Dong
67b6c696e9b901edeaf320d9
Pan Zhang
67b6c696e9b901edeaf320da
Yuhang Zang
admin_assigned
2025-02-20T16:07:11.149Z
63859cf3b2906edaf83af9f0
https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/mCgynEtoXdILvzyoPexii.png
Yuhang Zang
user
yuhangzang
67b6c696e9b901edeaf320db
Yuhang Cao
admin_assigned
2025-02-20T16:07:04.833Z
65000bef18830fabea469fdd
/avatars/b320c77dfad039d9f9c54127f610d44f.svg
Cao Yuhang
user
yhcao
67b6c696e9b901edeaf320dc
Dahua Lin
admin_assigned
2025-02-20T16:06:53.745Z
636317ed80c1a705a6eff396
/avatars/3db090e101b916d9256d0d3e043db71d.svg
Dahua Lin
user
lindahua
67b6c696e9b901edeaf320dd
Jiaqi Wang
admin_assigned
2025-02-20T16:06:47.581Z
64b4eec4faa3181a5eab9c46
/avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
Jiaqi Wang
user
myownskyW7
SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song
  Generation
Text-to-song generation, the task of creating vocals and accompaniment from
textual inputs, poses significant challenges due to domain complexity and data
scarcity. Existing approaches often employ multi-stage generation procedures,
resulting in cumbersome training and inference pipelines. In this paper, we
propose SongGen, a fully open-source, single-stage auto-regressive transformer
designed for controllable song generation. The proposed model facilitates
fine-grained control over diverse musical attributes, including lyrics and
textual descriptions of instrumentation, genre, mood, and timbre, while also
offering an optional three-second reference clip for voice cloning. Within a
unified auto-regressive framework, SongGen supports two output modes: mixed
mode, which generates a mixture of vocals and accompaniment directly, and
dual-track mode, which synthesizes them separately for greater flexibility in
downstream applications. We explore diverse token pattern strategies for each
mode, leading to notable improvements and valuable insights. Furthermore, we
design an automated data preprocessing pipeline with effective quality control.
To foster community engagement and future research, we will release our model
weights, training code, annotated data, and preprocessing pipeline. The
generated samples are showcased on our project page at
https://liuzh-19.github.io/SongGen/ , and the code will be available at
https://github.com/LiuZH-19/SongGen .
SongGen is a unified, single-stage, auto-regressive transformer model for text-to-song generation, offering controllable attributes and flexible output modes.
auto-regressive transformer multi-stage generation voice cloning mixed mode dual-track mode token pattern strategies automated data preprocessing pipeline