title: Controllable Music Production with Diffusion Models and Guidance
    Gradients
summary: We demonstrate how conditional generation from diffusion models can be used
  to tackle a variety of realistic tasks in the production of music in 44.1kHz
  stereo audio with sampling-time guidance. The scenarios we consider include
  continuation, inpainting and regeneration of musical audio, the creation of
  smooth transitions between two different music tracks, and the transfer of
  desired stylistic characteristics to existing audio clips. We achieve this by
  applying guidance at sampling time in a simple framework that supports both
  reconstruction and classification losses, or any combination of the two. This
  approach ensures that generated audio can match its surrounding context, or
  conform to a class distribution or latent representation specified relative to
  any suitable pre-trained classifier or embedding model.
paper_authors._id: 65430e2b4539d4807059985b
paper_authors.name: Mark Levy
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-02T13:38:53.856Z
paper_authors._id: 6543bd237a23df0e635d834c
paper_authors.avatarUrl: /avatars/d2f6869ab09d13d8f4628a73bcbbd023.svg
paper_authors.fullname: Mark Levy
paper_authors.type: user
paper_authors.user: gamboviol
paper_authors._id: 65430e2b4539d4807059985c
paper_authors.name: Bruno Di Giorgi
paper_authors._id: 65430e2b4539d4807059985d
paper_authors.name: Floris Weers
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-15T11:02:55.040Z
paper_authors._id: 65f41f1d6724ea19d7bc9052
paper_authors.avatarUrl: /avatars/4ccd6279aac3a18e1b23642a1a549e2f.svg
paper_authors.fullname: Floris Weers
paper_authors.type: user
paper_authors.user: fweers
paper_authors._id: 65430e2b4539d4807059985e
paper_authors.name: Angelos Katharopoulos
paper_authors._id: 65430e2b4539d4807059985f
paper_authors.name: Tom Nickson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-02T13:39:14.481Z
paper_authors._id: 62cdd89b3529c21a2285baca
paper_authors.avatarUrl: /avatars/1468fb8715bba441408b47ba834909dd.svg
paper_authors.fullname: Tom Nickson
paper_authors.type: user
paper_authors.user: TomNickson
paper_title: Controllable Music Production with Diffusion Models and Guidance
    Gradients
paper_summary: We demonstrate how conditional generation from diffusion models can be used
  to tackle a variety of realistic tasks in the production of music in 44.1kHz
  stereo audio with sampling-time guidance. The scenarios we consider include
  continuation, inpainting and regeneration of musical audio, the creation of
  smooth transitions between two different music tracks, and the transfer of
  desired stylistic characteristics to existing audio clips. We achieve this by
  applying guidance at sampling time in a simple framework that supports both
  reconstruction and classification losses, or any combination of the two. This
  approach ensures that generated audio can match its surrounding context, or
  conform to a class distribution or latent representation specified relative to
  any suitable pre-trained classifier or embedding model.
paper_ai_summary: Conditional generation from diffusion models is used for various music production tasks using sampling-time guidance, ensuring context matching and conforming to class distributions.
paper_ai_keywords: diffusion models conditional generation musical audio sampling-time guidance continuation inpainting regeneration smooth transitions stylistic characteristics reconstruction losses classification losses pre-trained classifier latent representation embedding model