title: YuE: Scaling Open Foundation Models for Long-Form Music Generation
summary: We tackle the task of long-form music generation--particularly the
  challenging lyrics-to-song problem--by introducing YuE, a family of
  open foundation models based on the LLaMA2 architecture. Specifically, YuE
  scales to trillions of tokens and generates up to five minutes of music while
  maintaining lyrical alignment, coherent musical structure, and engaging vocal
  melodies with appropriate accompaniment. It achieves this through (1)
  track-decoupled next-token prediction to overcome dense mixture signals, (2)
  structural progressive conditioning for long-context lyrical alignment, and (3)
  a multitask, multiphase pre-training recipe to converge and generalize. In
  addition, we redesign the in-context learning technique for music generation,
  enabling versatile style transfer (e.g., converting Japanese city pop into an
  English rap while preserving the original accompaniment) and bidirectional
  generation. Through extensive evaluation, we demonstrate that YuE matches or
  even surpasses some of the proprietary systems in musicality and vocal agility.
  In addition, fine-tuning YuE enables additional controls and enhanced support
  for tail languages. Furthermore, beyond generation, we show that YuE's learned
  representations can perform well on music understanding tasks, where the
  results of YuE match or exceed state-of-the-art methods on the MARBLE
  benchmark. Keywords: lyrics2song, song generation, long-form, foundation model,
  music generation
paper_authors._id: 67d1027435066eade61549ae
paper_authors.name: Ruibin Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:36:33.054Z
paper_authors._id: 5fd6f670053c8345eddc1b68
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5fd6f670053c8345eddc1b68/cuTsu2krRYHC6zYGD2dpQ.jpeg
paper_authors.fullname: Ruibin Yuan
paper_authors.type: user
paper_authors.user: a43992899
paper_authors._id: 67d1027435066eade61549af
paper_authors.name: Hanfeng Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:28:03.508Z
paper_authors._id: 638df7f3d975533b16a5a739
paper_authors.avatarUrl: /avatars/dbc8388e7c32c437b66619ea9b3ad827.svg
paper_authors.fullname: Hanfeng Lin
paper_authors.type: user
paper_authors.user: agent404
paper_authors._id: 67d1027435066eade61549b0
paper_authors.name: Shuyue Guo
paper_authors._id: 67d1027435066eade61549b1
paper_authors.name: Ge Zhang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-03-12T06:24:13.961Z
paper_authors._id: 638efcf4c67af472d316d424
paper_authors.avatarUrl: /avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
paper_authors.fullname: Ge Zhang
paper_authors.type: user
paper_authors.user: zhangysk
paper_authors._id: 67d1027435066eade61549b2
paper_authors.name: Jiahao Pan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-13T15:41:35.255Z
paper_authors._id: 647f7cf4b0e9676458a014f9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/647f7cf4b0e9676458a014f9/4L_0MbVErSmPYrk3mIzNP.jpeg
paper_authors.fullname: jiahao
paper_authors.type: user
paper_authors.user: cherishpjh
paper_authors._id: 67d1027435066eade61549b3
paper_authors.name: Yongyi Zang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:28:52.965Z
paper_authors._id: 63dda99c0f6d2d6c3efd2362
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63dda99c0f6d2d6c3efd2362/L5TfrxEWuwWDpR4GeuvI9.png
paper_authors.fullname: Yongyi Zang
paper_authors.type: user
paper_authors.user: yongyizang
paper_authors._id: 67d1027435066eade61549b4
paper_authors.name: Haohe Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:29:01.345Z
paper_authors._id: 6155245d1c762d4d61b51d5d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1677233708065-6155245d1c762d4d61b51d5d.png
paper_authors.fullname: haoheliu
paper_authors.type: user
paper_authors.user: haoheliu
paper_authors._id: 67d1027435066eade61549b5
paper_authors.name: Yiming Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:29:24.894Z
paper_authors._id: 6555e8d8a0c34cd61a6b9ce3
paper_authors.avatarUrl: /avatars/71dc562cef4bd42f6b762f036357c800.svg
paper_authors.fullname: yimingliang
paper_authors.type: user
paper_authors.user: yimingliang
paper_authors._id: 67d1027435066eade61549b6
paper_authors.name: Wenye Ma
paper_authors._id: 67d1027435066eade61549b7
paper_authors.name: Xingjian Du
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:36:36.330Z
paper_authors._id: 654907a4a1faff97850c4eff
paper_authors.avatarUrl: /avatars/458c90151614bc7f116943b6e67d6b8a.svg
paper_authors.fullname: du
paper_authors.type: user
paper_authors.user: dododododo
paper_authors._id: 67d1027435066eade61549b8
paper_authors.name: Xinrun Du
paper_authors._id: 67d1027435066eade61549b9
paper_authors.name: Zhen Ye
paper_authors._id: 67d1027435066eade61549ba
paper_authors.name: Tianyu Zheng
paper_authors._id: 67d1027435066eade61549bb
paper_authors.name: Yinghao Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:30:29.451Z
paper_authors._id: 6410665d5364a661bee22524
paper_authors.avatarUrl: /avatars/f1cb0e07f36933187ceccbd5dcbeff79.svg
paper_authors.fullname: Yinghao Ma
paper_authors.type: user
paper_authors.user: nicolaus625
paper_authors._id: 67d1027435066eade61549bc
paper_authors.name: Minghao Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:36:39.193Z
paper_authors._id: 6417d9ea8f689506e7148417
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6417d9ea8f689506e7148417/bAYcruWNw4WvmuQcGgcwC.jpeg
paper_authors.fullname: minghao
paper_authors.type: user
paper_authors.user: Liam-Liu
paper_authors._id: 67d1027435066eade61549bd
paper_authors.name: Zeyue Tian
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-16T21:17:50.589Z
paper_authors._id: 6504409b52ca06fef957266d
paper_authors.avatarUrl: /avatars/ac96eab7318cd8d472601cf7e7c50c97.svg
paper_authors.fullname: Zeyue
paper_authors.type: user
paper_authors.user: Zeyue7
paper_authors._id: 67d1027435066eade61549be
paper_authors.name: Ziya Zhou
paper_authors._id: 67d1027435066eade61549bf
paper_authors.name: Liumeng Xue
paper_authors._id: 67d1027435066eade61549c0
paper_authors.name: Xingwei Qu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-03-12T03:41:43.139Z
paper_authors._id: 6628adb14277eae0da5eee28
paper_authors.avatarUrl: /avatars/6cb41b80cc5e014e455dfc2a22682e64.svg
paper_authors.fullname: HKUST Audio
paper_authors.type: user
paper_authors.user: HKUST-Audio
paper_authors._id: 67d1027435066eade61549c1
paper_authors.name: Yizhi Li
paper_authors._id: 67d1027435066eade61549c2
paper_authors.name: Shangda Wu
paper_authors._id: 67d1027435066eade61549c3
paper_authors.name: Tianhao Shen
paper_authors._id: 67d1027435066eade61549c4
paper_authors.name: Ziyang Ma
paper_authors._id: 67d1027435066eade61549c5
paper_authors.name: Jun Zhan
paper_authors._id: 67d1027435066eade61549c6
paper_authors.name: Chunhui Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:31:34.824Z
paper_authors._id: 66610c2ec629e0e089fc9cf9
paper_authors.avatarUrl: /avatars/b3a958de94ba93a8b21ae6d1011f0b45.svg
paper_authors.fullname: Chun-Hui Wang
paper_authors.type: user
paper_authors.user: ChunHuiWangFN
paper_authors._id: 67d1027435066eade61549c7
paper_authors.name: Yatian Wang
paper_authors._id: 67d1027435066eade61549c8
paper_authors.name: Xiaowei Chi
paper_authors._id: 67d1027435066eade61549c9
paper_authors.name: Xinyue Zhang
paper_authors._id: 67d1027435066eade61549ca
paper_authors.name: Zhenzhu Yang
paper_authors._id: 67d1027435066eade61549cb
paper_authors.name: Xiangzhou Wang
paper_authors._id: 67d1027435066eade61549cc
paper_authors.name: Shansong Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-28T08:38:25.256Z
paper_authors._id: 64c1f3560a2545084f08350b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64c1f3560a2545084f08350b/RbSzN5eFvExrfRLX1Xi_g.jpeg
paper_authors.fullname: Shansong Liu
paper_authors.type: user
paper_authors.user: dadinghh2
paper_authors._id: 67d1027435066eade61549cd
paper_authors.name: Lingrui Mei
paper_authors._id: 67d1027435066eade61549ce
paper_authors.name: Peng Li
paper_authors._id: 67d1027435066eade61549cf
paper_authors.name: Junjie Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T14:25:45.882Z
paper_authors._id: 62579c55b98dcaa7e0de285d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62579c55b98dcaa7e0de285d/0YUd5nloul_bW9yolDGGo.jpeg
paper_authors.fullname: wangjunjie
paper_authors.type: user
paper_authors.user: wanng
paper_authors._id: 67d1027435066eade61549d0
paper_authors.name: Jianwei Yu
paper_authors._id: 67d1027435066eade61549d1
paper_authors.name: Guojian Pang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:34:44.025Z
paper_authors._id: 6454761288c97c8796dcbcee
paper_authors.avatarUrl: /avatars/ad39352af72829d687355c92cc8e74a4.svg
paper_authors.fullname: Guojian Pang
paper_authors.type: user
paper_authors.user: WabbyWabbo
paper_authors._id: 67d1027435066eade61549d2
paper_authors.name: Xu Li
paper_authors._id: 67d1027435066eade61549d3
paper_authors.name: Zihao Wang
paper_authors._id: 67d1027435066eade61549d4
paper_authors.name: Xiaohuan Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:33:44.178Z
paper_authors._id: 668f5875b5b3081d776e4094
paper_authors.avatarUrl: /avatars/8c763393f25afbe5fb8b132f775e746a.svg
paper_authors.fullname: Xiaohuan Zhou
paper_authors.type: user
paper_authors.user: XiaohuanZhou
paper_authors._id: 67d1027435066eade61549d5
paper_authors.name: Lijun Yu
paper_authors._id: 67d1027435066eade61549d6
paper_authors.name: Emmanouil Benetos
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:34:09.275Z
paper_authors._id: 66b0a0839797b94f001ed874
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66b0a0839797b94f001ed874/BBtlJdQNyzQX10chnU81s.jpeg
paper_authors.fullname: Emmanouil Benetos
paper_authors.type: user
paper_authors.user: emmanouilb
paper_authors._id: 67d1027435066eade61549d7
paper_authors.name: Yong Chen
paper_authors._id: 67d1027435066eade61549d8
paper_authors.name: Chenghua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:34:33.279Z
paper_authors._id: 6442ba123610a28a4ad5382f
paper_authors.avatarUrl: /avatars/97f2d3d2c10058ea41d7bf9087e1f619.svg
paper_authors.fullname: Chenghua Lin
paper_authors.type: user
paper_authors.user: chenghualin
paper_authors._id: 67d1027435066eade61549d9
paper_authors.name: Xie Chen
paper_authors._id: 67d1027435066eade61549da
paper_authors.name: Gus Xia
paper_authors._id: 67d1027435066eade61549db
paper_authors.name: Zhaoxiang Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:33:32.186Z
paper_authors._id: 6588d5711b44d0e6942e8489
paper_authors.avatarUrl: /avatars/f6624b46071e4d1548ea719f26b90b17.svg
paper_authors.fullname: zhangzhaoxiang
paper_authors.type: user
paper_authors.user: zhangzhaoxiang
paper_authors._id: 67d1027435066eade61549dc
paper_authors.name: Chao Zhang
paper_authors._id: 67d1027435066eade61549dd
paper_authors.name: Wenhu Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:33:13.185Z
paper_authors._id: 6313a86154e6e5d9f0f94e04
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1662232951344-6313a86154e6e5d9f0f94e04.jpeg
paper_authors.fullname: Wenhu Chen
paper_authors.type: user
paper_authors.user: wenhu
paper_authors._id: 67d1027435066eade61549de
paper_authors.name: Xinyu Zhou
paper_authors._id: 67d1027435066eade61549df
paper_authors.name: Xipeng Qiu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:32:39.488Z
paper_authors._id: 61457b8deff2c9fdb4de4988
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1632381702899-61457b8deff2c9fdb4de4988.jpeg
paper_authors.fullname: Xipeng Qiu
paper_authors.type: user
paper_authors.user: xpqiu
paper_authors._id: 67d1027435066eade61549e0
paper_authors.name: Roger Dannenberg
paper_authors._id: 67d1027435066eade61549e1
paper_authors.name: Jiaheng Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:32:23.034Z
paper_authors._id: 65377c30e48353201e6fdda0
paper_authors.avatarUrl: /avatars/a8f803b6f2e598eaee9c52c0d2ddfc16.svg
paper_authors.fullname: Jiaheng Liu
paper_authors.type: user
paper_authors.user: CheeryLJH
paper_authors._id: 67d1027435066eade61549e2
paper_authors.name: Jian Yang
paper_authors._id: 67d1027435066eade61549e3
paper_authors.name: Wenhao Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:32:09.795Z
paper_authors._id: 641e5bf65f274a0a92c2f6a2
paper_authors.avatarUrl: /avatars/c15a54c51998c0e6367685e8e1737ec9.svg
paper_authors.fullname: Wenhao Huang
paper_authors.type: user
paper_authors.user: EZ-hwh
paper_authors._id: 67d1027435066eade61549e4
paper_authors.name: Wei Xue
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:31:58.392Z
paper_authors._id: 647403b971f07ae738d4d1dc
paper_authors.avatarUrl: /avatars/13a6ff75560c18d857d748cde0feedf3.svg
paper_authors.fullname: weixue
paper_authors.type: user
paper_authors.user: weixue
paper_authors._id: 67d1027435066eade61549e5
paper_authors.name: Xu Tan
paper_authors._id: 67d1027435066eade61549e6
paper_authors.name: Yike Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:31:46.078Z
paper_authors._id: 64762b325f70f9b2d0ade28e
paper_authors.avatarUrl: /avatars/1f37382724b475ece805f943a8858acd.svg
paper_authors.fullname: Yike Guo
paper_authors.type: user
paper_authors.user: SuaLily
paper_title: YuE: Scaling Open Foundation Models for Long-Form Music Generation
paper_summary: We tackle the task of long-form music generation--particularly the
  challenging lyrics-to-song problem--by introducing YuE, a family of
  open foundation models based on the LLaMA2 architecture. Specifically, YuE
  scales to trillions of tokens and generates up to five minutes of music while
  maintaining lyrical alignment, coherent musical structure, and engaging vocal
  melodies with appropriate accompaniment. It achieves this through (1)
  track-decoupled next-token prediction to overcome dense mixture signals, (2)
  structural progressive conditioning for long-context lyrical alignment, and (3)
  a multitask, multiphase pre-training recipe to converge and generalize. In
  addition, we redesign the in-context learning technique for music generation,
  enabling versatile style transfer (e.g., converting Japanese city pop into an
  English rap while preserving the original accompaniment) and bidirectional
  generation. Through extensive evaluation, we demonstrate that YuE matches or
  even surpasses some of the proprietary systems in musicality and vocal agility.
  In addition, fine-tuning YuE enables additional controls and enhanced support
  for tail languages. Furthermore, beyond generation, we show that YuE's learned
  representations can perform well on music understanding tasks, where the
  results of YuE match or exceed state-of-the-art methods on the MARBLE
  benchmark. Keywords: lyrics2song, song generation, long-form, foundation model,
  music generation
paper_ai_summary: YuE, a family of open foundation models based on LLaMA2, can generate long-form music with aligned lyrics, coherent structure, and appropriate accompaniment using innovative techniques in next-token prediction, conditioning, and pre-training.
paper_ai_keywords: track-decoupled next-token prediction structural progressive conditioning multitask multiphase pre-training in-context learning style transfer bidirectional generation music generation music understanding MARBLE benchmark