Arrange, Inpaint, and Refine: Steerable Long-term Music Audio Generation and Editing via Content-based Controls
<|reference_start|>Arrange, Inpaint, and Refine: Steerable Long-term Music Audio Generation and Editing via Content-based Controls: Controllable music generation plays a vital role in human-AI music co-creation. While Large Language Models (LLMs) have shown promise in generating high-quality music, their focus on autoregressive generation limits their utility in music editing tasks. To address this gap, we propose a novel approach leveraging a parameter-efficient heterogeneous adapter combined with a masking training scheme. This approach enables autoregressive language models to seamlessly address music inpainting tasks. Additionally, our method integrates frame-level content-based controls, facilitating track-conditioned music refinement and score-conditioned music arrangement. We apply this method to fine-tune MusicGen, a leading autoregressive music generation model. Our experiments demonstrate promising results across multiple music editing tasks, offering more flexible controls for future AI-driven music editing tools. The source codes and a demo page showcasing our work are available at https://kikyo-16.github.io/AIR.<|reference_end|>
@article{lin2024arrange,,
    title={Arrange, Inpaint, and Refine: Steerable Long-term Music Audio Generation
  and Editing via Content-based Controls},
    author={Liwei Lin, Gus Xia, Yixiao Zhang, Junyan Jiang},
    journal={arXiv preprint arXiv:2402.09508},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.09508},
    primaryClass={cs.SD cs.AI eess.AS}
}