title: VampNet: Music Generation via Masked Acoustic Token Modeling
summary: We introduce VampNet, a masked acoustic token modeling approach to music
  synthesis, compression, inpainting, and variation. We use a variable masking
  schedule during training which allows us to sample coherent music from the
  model by applying a variety of masking approaches (called prompts) during
  inference. VampNet is non-autoregressive, leveraging a bidirectional
  transformer architecture that attends to all tokens in a forward pass. With
  just 36 sampling passes, VampNet can generate coherent high-fidelity musical
  waveforms. We show that by prompting VampNet in various ways, we can apply it
  to tasks like music compression, inpainting, outpainting, continuation, and
  looping with variation (vamping). Appropriately prompted, VampNet is capable of
  maintaining style, genre, instrumentation, and other high-level aspects of the
  music. This flexible prompting capability makes VampNet a powerful music
  co-creation tool. Code and audio samples are available online.
paper_authors._id: 64acc3f1f5d40283c620a0fe
paper_authors.name: Hugo Flores Garcia
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-12T07:29:30.137Z
paper_authors._id: 60d3a674b8448e1785bbda24
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60d3a674b8448e1785bbda24/8BS8q0VqI6baD0hRMSg6T.jpeg
paper_authors.fullname: Hugo Flores
paper_authors.type: user
paper_authors.user: hugggof
paper_authors._id: 64acc3f1f5d40283c620a0ff
paper_authors.name: Prem Seetharaman
paper_authors._id: 64acc3f1f5d40283c620a100
paper_authors.name: Rithesh Kumar
paper_authors._id: 64acc3f1f5d40283c620a101
paper_authors.name: Bryan Pardo
paper_title: VampNet: Music Generation via Masked Acoustic Token Modeling
paper_summary: We introduce VampNet, a masked acoustic token modeling approach to music
  synthesis, compression, inpainting, and variation. We use a variable masking
  schedule during training which allows us to sample coherent music from the
  model by applying a variety of masking approaches (called prompts) during
  inference. VampNet is non-autoregressive, leveraging a bidirectional
  transformer architecture that attends to all tokens in a forward pass. With
  just 36 sampling passes, VampNet can generate coherent high-fidelity musical
  waveforms. We show that by prompting VampNet in various ways, we can apply it
  to tasks like music compression, inpainting, outpainting, continuation, and
  looping with variation (vamping). Appropriately prompted, VampNet is capable of
  maintaining style, genre, instrumentation, and other high-level aspects of the
  music. This flexible prompting capability makes VampNet a powerful music
  co-creation tool. Code and audio samples are available online.
paper_ai_keywords: masked acoustic token modeling VampNet variable masking schedule prompts non-autoregressive bidirectional transformer architecture sampling passes high-fidelity musical waveforms music compression music inpainting outpainting continuation looping vamping style genre instrumentation high-level aspects music co-creation tool