Fast Timing-Conditioned Latent Audio Diffusion
Generating long-form 44.1kHz stereo audio from text prompts can be
computationally demanding. Further, most previous works do not tackle that
music and sound effects naturally vary in their duration. Our research focuses
on the efficient generation of long-form, variable-length stereo music and
sounds at 44.1kHz using text prompts with a generative model. Stable Audio is
based on latent diffusion, with its latent defined by a fully-convolutional
variational autoencoder. It is conditioned on text prompts as well as timing
embeddings, allowing for fine control over both the content and length of the
generated music and sounds. Stable Audio is capable of rendering stereo signals
of up to 95 sec at 44.1kHz in 8 sec on an A100 GPU. Despite its compute
efficiency and fast inference, it is one of the best in two public
text-to-music and -audio benchmarks and, differently from state-of-the-art
models, can generate music with structure and stereo sounds.
65c463ca6b793334efab148c
Zach Evans
extracted_confirmed
2024-02-08T18:21:16.847Z
6306bca3cb5492c985a30588
/avatars/c4ab7a409d1d212aa6ac04e8c2632655.svg
Zach Evans
user
Fauno15
65c463ca6b793334efab148d
CJ Carr
admin_assigned
2024-02-08T11:51:24.248Z
64932b4a7ed33d93550902bb
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/RAF-iCFqukRnjZ1VbtVZU.jpeg
CJ Carr
user
cjus13
65c463ca6b793334efab148e
Josiah Taylor
admin_assigned
2024-02-08T11:51:33.200Z
642c16a0c3694d2b74558f14
https://cdn-avatars.huggingface.co/v1/production/uploads/642c16a0c3694d2b74558f14/ILnrIBeO_tA3BjkEmyyFZ.png
Josiah Taylor
user
AtOdds
65c463ca6b793334efab148f
Scott H. Hawley
admin_assigned
2024-02-08T11:51:43.230Z
608ce15acbb3288dd7faf8e7
https://cdn-avatars.huggingface.co/v1/production/uploads/608ce15acbb3288dd7faf8e7/8blAfMJirdA_UwjbargH-.jpeg
Scott Hawley
user
drscotthawley
65c463ca6b793334efab1490
Jordi Pons
Fast Timing-Conditioned Latent Audio Diffusion
Generating long-form 44.1kHz stereo audio from text prompts can be
computationally demanding. Further, most previous works do not tackle that
music and sound effects naturally vary in their duration. Our research focuses
on the efficient generation of long-form, variable-length stereo music and
sounds at 44.1kHz using text prompts with a generative model. Stable Audio is
based on latent diffusion, with its latent defined by a fully-convolutional
variational autoencoder. It is conditioned on text prompts as well as timing
embeddings, allowing for fine control over both the content and length of the
generated music and sounds. Stable Audio is capable of rendering stereo signals
of up to 95 sec at 44.1kHz in 8 sec on an A100 GPU. Despite its compute
efficiency and fast inference, it is one of the best in two public
text-to-music and -audio benchmarks and, differently from state-of-the-art
models, can generate music with structure and stereo sounds.
Stable Audio uses a latent diffusion model with a fully-convolutional variational autoencoder to efficiently generate high-quality, variable-length stereo audio from text prompts.
latent diffusion fully-convolutional variational autoencoder text prompts timing embeddings stereo audio text-to-music text-to-audio benchmarks