title: Music Style Transfer with Time-Varying Inversion of Diffusion Models
summary: With the development of diffusion models, text-guided image style transfer
  has demonstrated high-quality controllable synthesis results. However, the
  utilization of text for diverse music style transfer poses significant
  challenges, primarily due to the limited availability of matched audio-text
  datasets. Music, being an abstract and complex art form, exhibits variations
  and intricacies even within the same genre, thereby making accurate textual
  descriptions challenging. This paper presents a music style transfer approach
  that effectively captures musical attributes using minimal data. We introduce a
  novel time-varying textual inversion module to precisely capture
  mel-spectrogram features at different levels. During inference, we propose a
  bias-reduced stylization technique to obtain stable results. Experimental
  results demonstrate that our method can transfer the style of specific
  instruments, as well as incorporate natural sounds to compose melodies. Samples
  and source code are available at https://lsfhuihuiff.github.io/MusicTI/.
paper_authors._id: 65d757049103d35b77b0b8ea
paper_authors.name: Sifei Li
paper_authors._id: 65d757049103d35b77b0b8eb
paper_authors.name: Yuxin Zhang
paper_authors._id: 65d757049103d35b77b0b8ec
paper_authors.name: Fan Tang
paper_authors._id: 65d757049103d35b77b0b8ed
paper_authors.name: Chongyang Ma
paper_authors._id: 65d757049103d35b77b0b8ee
paper_authors.name: Weiming dong
paper_authors._id: 65d757049103d35b77b0b8ef
paper_authors.name: Changsheng Xu
paper_title: Music Style Transfer with Time-Varying Inversion of Diffusion Models
paper_summary: With the development of diffusion models, text-guided image style transfer
  has demonstrated high-quality controllable synthesis results. However, the
  utilization of text for diverse music style transfer poses significant
  challenges, primarily due to the limited availability of matched audio-text
  datasets. Music, being an abstract and complex art form, exhibits variations
  and intricacies even within the same genre, thereby making accurate textual
  descriptions challenging. This paper presents a music style transfer approach
  that effectively captures musical attributes using minimal data. We introduce a
  novel time-varying textual inversion module to precisely capture
  mel-spectrogram features at different levels. During inference, we propose a
  bias-reduced stylization technique to obtain stable results. Experimental
  results demonstrate that our method can transfer the style of specific
  instruments, as well as incorporate natural sounds to compose melodies. Samples
  and source code are available at https://lsfhuihuiff.github.io/MusicTI/.
paper_ai_keywords: diffusion models text-guided image style transfer music style transfer matched audio-text datasets mel-spectrogram features time-varying textual inversion module bias-reduced stylization technique