title: Versatile Framework for Song Generation with Prompt-based Control
summary: Song generation focuses on producing controllable high-quality songs based on
  various prompts. However, existing methods struggle to generate vocals and
  accompaniments with prompt-based control and proper alignment. Additionally,
  they fall short in supporting various tasks. To address these challenges, we
  introduce VersBand, a multi-task song generation framework for synthesizing
  high-quality, aligned songs with prompt-based control. VersBand comprises these
  primary models: 1) VocalBand, a decoupled model, leverages the flow-matching
  method for generating singing styles, pitches, and mel-spectrograms, allowing
  fast, high-quality vocal generation with style control. 2) AccompBand, a
  flow-based transformer model, incorporates the Band-MOE, selecting suitable
  experts for enhanced quality, alignment, and control. This model allows for
  generating controllable, high-quality accompaniments aligned with vocals. 3)
  Two generation models, LyricBand for lyrics and MelodyBand for melodies,
  contribute to the comprehensive multi-task song generation system, allowing for
  extensive control based on multiple prompts. Experimental results demonstrate
  that VersBand performs better over baseline models across multiple song
  generation tasks using objective and subjective metrics. Audio samples are
  available at https://VersBand.github.io.
paper_authors._id: 6810d9d76d704ff9ea87ab82
paper_authors.name: Yu Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-30T09:56:44.960Z
paper_authors._id: 66569729ea21cfae5f5797c4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66569729ea21cfae5f5797c4/IguwJzljFN3QiEd1bn5BP.jpeg
paper_authors.fullname: Yu Zhang
paper_authors.type: user
paper_authors.user: AaronZ345
paper_authors._id: 6810d9d76d704ff9ea87ab83
paper_authors.name: Wenxiang Guo
paper_authors._id: 6810d9d76d704ff9ea87ab84
paper_authors.name: Changhao Pan
paper_authors._id: 6810d9d76d704ff9ea87ab85
paper_authors.name: Zhiyuan Zhu
paper_authors._id: 6810d9d76d704ff9ea87ab86
paper_authors.name: Ruiqi Li
paper_authors._id: 6810d9d76d704ff9ea87ab87
paper_authors.name: Jingyu Lu
paper_authors._id: 6810d9d76d704ff9ea87ab88
paper_authors.name: Rongjie Huang
paper_authors._id: 6810d9d76d704ff9ea87ab89
paper_authors.name: Ruiyuan Zhang
paper_authors._id: 6810d9d76d704ff9ea87ab8a
paper_authors.name: Zhiqing Hong
paper_authors._id: 6810d9d76d704ff9ea87ab8b
paper_authors.name: Ziyue Jiang
paper_authors._id: 6810d9d76d704ff9ea87ab8c
paper_authors.name: Zhou Zhao
paper_title: Versatile Framework for Song Generation with Prompt-based Control
paper_summary: Song generation focuses on producing controllable high-quality songs based on
  various prompts. However, existing methods struggle to generate vocals and
  accompaniments with prompt-based control and proper alignment. Additionally,
  they fall short in supporting various tasks. To address these challenges, we
  introduce VersBand, a multi-task song generation framework for synthesizing
  high-quality, aligned songs with prompt-based control. VersBand comprises these
  primary models: 1) VocalBand, a decoupled model, leverages the flow-matching
  method for generating singing styles, pitches, and mel-spectrograms, allowing
  fast, high-quality vocal generation with style control. 2) AccompBand, a
  flow-based transformer model, incorporates the Band-MOE, selecting suitable
  experts for enhanced quality, alignment, and control. This model allows for
  generating controllable, high-quality accompaniments aligned with vocals. 3)
  Two generation models, LyricBand for lyrics and MelodyBand for melodies,
  contribute to the comprehensive multi-task song generation system, allowing for
  extensive control based on multiple prompts. Experimental results demonstrate
  that VersBand performs better over baseline models across multiple song
  generation tasks using objective and subjective metrics. Audio samples are
  available at https://VersBand.github.io.
paper_ai_summary: VersBand is a multi-task song generation framework that integrates multiple models for high-quality, aligned vocals and accompaniments with prompt-based control.
paper_ai_keywords: flow-matching flow-based transformer Band-MOE multi-task song generation vocal generation accompaniment generation lyrics generation melody generation