StemGen: A music generation model that listens
End-to-end generation of musical audio using deep learning techniques has
seen an explosion of activity recently. However, most models concentrate on
generating fully mixed music in response to abstract conditioning information.
In this work, we present an alternative paradigm for producing music generation
models that can listen and respond to musical context. We describe how such a
model can be constructed using a non-autoregressive, transformer-based model
architecture and present a number of novel architectural and sampling
improvements. We train the described architecture on both an open-source and a
proprietary dataset. We evaluate the produced models using standard quality
metrics and a new approach based on music information retrieval descriptors.
The resulting model reaches the audio quality of state-of-the-art
text-conditioned models, as well as exhibiting strong musical coherence with
its context.
657bd8f11fa4e4e152c12793
Julian D. Parker
claimed_verified
2023-12-15T10:43:28.573Z
657c2cf25c6f0b1f36077033
/avatars/af538ee6d185d49efc0b7cf8660217ad.svg
Julian Parker
user
julian-parker
657bd8f11fa4e4e152c12794
Janne Spijkervet
657bd8f11fa4e4e152c12795
Katerina Kosta
claimed_verified
2023-12-15T12:53:27.574Z
657c40e664dab2f4c90d6071
/avatars/cd960f8bad12b18ee23271303aff5eb1.svg
Katerina Kosta
user
katkost
657bd8f11fa4e4e152c12796
Furkan Yesiler
657bd8f11fa4e4e152c12797
Boris Kuznetsov
657bd8f11fa4e4e152c12798
Ju-Chiang Wang
657bd8f11fa4e4e152c12799
Matt Avent
admin_assigned
2023-12-15T10:06:58.538Z
63de3c9915266dd945f0d3d0
/avatars/bebe36499f0e1f2482e9323d19e0fa90.svg
Matt Avent
user
matildah
657bd8f11fa4e4e152c1279a
Jitong Chen
657bd8f11fa4e4e152c1279b
Duc Le
StemGen: A music generation model that listens
End-to-end generation of musical audio using deep learning techniques has
seen an explosion of activity recently. However, most models concentrate on
generating fully mixed music in response to abstract conditioning information.
In this work, we present an alternative paradigm for producing music generation
models that can listen and respond to musical context. We describe how such a
model can be constructed using a non-autoregressive, transformer-based model
architecture and present a number of novel architectural and sampling
improvements. We train the described architecture on both an open-source and a
proprietary dataset. We evaluate the produced models using standard quality
metrics and a new approach based on music information retrieval descriptors.
The resulting model reaches the audio quality of state-of-the-art
text-conditioned models, as well as exhibiting strong musical coherence with
its context.
A transformer-based, non-autoregressive model generates musically coherent audio by responding to context, achieving audio quality comparable to text-conditioned models.
non-autoregressive transformer-based music generation musical coherence music information retrieval descriptors state-of-the-art text-conditioned models