Long-form music generation with latent diffusion
Audio-based generative models for music have seen great strides recently, but
so far have not managed to produce full-length music tracks with coherent
musical structure. We show that by training a generative model on long temporal
contexts it is possible to produce long-form music of up to 4m45s. Our model
consists of a diffusion-transformer operating on a highly downsampled
continuous latent representation (latent rate of 21.5Hz). It obtains
state-of-the-art generations according to metrics on audio quality and prompt
alignment, and subjective tests reveal that it produces full-length music with
coherent structure.
66200147b760cd2e987fc32a
Zach Evans
66200147b760cd2e987fc32b
Julian D. Parker
claimed_verified
2024-04-18T08:09:49.408Z
657c2cf25c6f0b1f36077033
/avatars/af538ee6d185d49efc0b7cf8660217ad.svg
Julian Parker
user
julian-parker
66200147b760cd2e987fc32c
CJ Carr
admin_assigned
2024-04-18T08:19:53.221Z
64932b4a7ed33d93550902bb
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/RAF-iCFqukRnjZ1VbtVZU.jpeg
CJ Carr
user
cjus13
66200147b760cd2e987fc32d
Zack Zukowski
66200147b760cd2e987fc32e
Josiah Taylor
admin_assigned
2024-04-18T08:20:14.709Z
642c16a0c3694d2b74558f14
https://cdn-avatars.huggingface.co/v1/production/uploads/642c16a0c3694d2b74558f14/ILnrIBeO_tA3BjkEmyyFZ.png
Josiah Taylor
user
AtOdds
66200147b760cd2e987fc32f
Jordi Pons
Long-form music generation with latent diffusion
Audio-based generative models for music have seen great strides recently, but
so far have not managed to produce full-length music tracks with coherent
musical structure. We show that by training a generative model on long temporal
contexts it is possible to produce long-form music of up to 4m45s. Our model
consists of a diffusion-transformer operating on a highly downsampled
continuous latent representation (latent rate of 21.5Hz). It obtains
state-of-the-art generations according to metrics on audio quality and prompt
alignment, and subjective tests reveal that it produces full-length music with
coherent structure.
A diffusion-transformer trained on long temporal contexts can generate full-length music tracks with coherent structure at a latent rate of 21.5Hz.
diffusion-transformer continuous latent representation