title: Instruct-MusicGen: Unlocking Text-to-Music Editing for Music Language
    Models via Instruction Tuning
summary: Recent advances in text-to-music editing, which employ text queries to modify
  music (e.g.\ by changing its style or adjusting instrumental components),
  present unique challenges and opportunities for AI-assisted music creation.
  Previous approaches in this domain have been constrained by the necessity to
  train specific editing models from scratch, which is both resource-intensive
  and inefficient; other research uses large language models to predict edited
  music, resulting in imprecise audio reconstruction. To Combine the strengths
  and address these limitations, we introduce Instruct-MusicGen, a novel approach
  that finetunes a pretrained MusicGen model to efficiently follow editing
  instructions such as adding, removing, or separating stems. Our approach
  involves a modification of the original MusicGen architecture by incorporating
  a text fusion module and an audio fusion module, which allow the model to
  process instruction texts and audio inputs concurrently and yield the desired
  edited music. Remarkably, Instruct-MusicGen only introduces 8% new parameters
  to the original MusicGen model and only trains for 5K steps, yet it achieves
  superior performance across all tasks compared to existing baselines, and
  demonstrates performance comparable to the models trained for specific tasks.
  This advancement not only enhances the efficiency of text-to-music editing but
  also broadens the applicability of music language models in dynamic music
  production environments.
paper_authors._id: 6656afa1b7e6a9f09553e291
paper_authors.name: Yixiao Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-29T08:23:39.716Z
paper_authors._id: 6376a5aad49217e154f580aa
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1668720030380-noauth.jpeg
paper_authors.fullname: Yixiao ZHang
paper_authors.type: user
paper_authors.user: ldzhangyx
paper_authors._id: 6656afa1b7e6a9f09553e292
paper_authors.name: Yukara Ikemiya
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-29T08:23:45.761Z
paper_authors._id: 62fbbdcd21c444a56f7a6159
paper_authors.avatarUrl: /avatars/3f83effd52773f0806f58331d8734861.svg
paper_authors.fullname: Yukara Ikemiya
paper_authors.type: user
paper_authors.user: yukara13
paper_authors._id: 6656afa1b7e6a9f09553e293
paper_authors.name: Woosung Choi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-29T08:24:36.131Z
paper_authors._id: 62c5411c9f9e408c41073c3e
paper_authors.avatarUrl: /avatars/2e046d996af8441127a38ba23a6248e8.svg
paper_authors.fullname: Woosung Choi
paper_authors.type: user
paper_authors.user: Woosung
paper_authors._id: 6656afa1b7e6a9f09553e294
paper_authors.name: Naoki Murata
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-29T08:24:42.188Z
paper_authors._id: 63e1e9627fbb6ae4d4f59d76
paper_authors.avatarUrl: /avatars/22197a8c85232135609331766299c3e7.svg
paper_authors.fullname: Naoki Murata
paper_authors.type: user
paper_authors.user: muttyjp
paper_authors._id: 6656afa1b7e6a9f09553e295
paper_authors.name: Marco A. Martínez-Ramírez
paper_authors._id: 6656afa1b7e6a9f09553e296
paper_authors.name: Liwei Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-29T08:25:15.656Z
paper_authors._id: 66324a8cfebe847d074da206
paper_authors.avatarUrl: /avatars/a9498b2fb36c4508dacbf61a4458e2dd.svg
paper_authors.fullname: Liwei Lin
paper_authors.type: user
paper_authors.user: Kikyo-16
paper_authors._id: 6656afa1b7e6a9f09553e297
paper_authors.name: Gus Xia
paper_authors._id: 6656afa1b7e6a9f09553e298
paper_authors.name: Wei-Hsiang Liao
paper_authors._id: 6656afa1b7e6a9f09553e299
paper_authors.name: Yuki Mitsufuji
paper_authors._id: 6656afa1b7e6a9f09553e29a
paper_authors.name: Simon Dixon
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-29T08:26:01.380Z
paper_authors._id: 643ac28aff50448bcfc24b19
paper_authors.avatarUrl: /avatars/18d201b94829bae9985dbb4ec618e1ff.svg
paper_authors.fullname: Simon Dixon
paper_authors.type: user
paper_authors.user: Dexdixon
paper_title: Instruct-MusicGen: Unlocking Text-to-Music Editing for Music Language
    Models via Instruction Tuning
paper_summary: Recent advances in text-to-music editing, which employ text queries to modify
  music (e.g.\ by changing its style or adjusting instrumental components),
  present unique challenges and opportunities for AI-assisted music creation.
  Previous approaches in this domain have been constrained by the necessity to
  train specific editing models from scratch, which is both resource-intensive
  and inefficient; other research uses large language models to predict edited
  music, resulting in imprecise audio reconstruction. To Combine the strengths
  and address these limitations, we introduce Instruct-MusicGen, a novel approach
  that finetunes a pretrained MusicGen model to efficiently follow editing
  instructions such as adding, removing, or separating stems. Our approach
  involves a modification of the original MusicGen architecture by incorporating
  a text fusion module and an audio fusion module, which allow the model to
  process instruction texts and audio inputs concurrently and yield the desired
  edited music. Remarkably, Instruct-MusicGen only introduces 8% new parameters
  to the original MusicGen model and only trains for 5K steps, yet it achieves
  superior performance across all tasks compared to existing baselines, and
  demonstrates performance comparable to the models trained for specific tasks.
  This advancement not only enhances the efficiency of text-to-music editing but
  also broadens the applicability of music language models in dynamic music
  production environments.
paper_ai_summary: Instruct-MusicGen, a finetuned MusicGen model with integrated text and audio modules, efficiently follows editing instructions with minimal additional parameters and achieves superior performance in text-to-music editing.
paper_ai_keywords: MusicGen text fusion module audio fusion module text-to-music editing parameter-efficient fine-tuning editing instructions task-specific training audio reconstruction dynamic music production