Music Consistency Models
Consistency models have exhibited remarkable capabilities in facilitating
efficient image/video generation, enabling synthesis with minimal sampling
steps. It has proven to be advantageous in mitigating the computational burdens
associated with diffusion models. Nevertheless, the application of consistency
models in music generation remains largely unexplored. To address this gap, we
present Music Consistency Models (MusicCM), which leverages the
concept of consistency models to efficiently synthesize mel-spectrogram for
music clips, maintaining high quality while minimizing the number of sampling
steps. Building upon existing text-to-music diffusion models, the
MusicCM model incorporates consistency distillation and adversarial
discriminator training. Moreover, we find it beneficial to generate extended
coherent music by incorporating multiple diffusion processes with shared
constraints. Experimental results reveal the effectiveness of our model in
terms of computational efficiency, fidelity, and naturalness. Notable,
MusicCM achieves seamless music synthesis with a mere four sampling
steps, e.g., only one second per minute of the music clip, showcasing the
potential for real-time application.
66271377b24310dd5f0c3a8f
Zhengcong Fei
admin_assigned
2024-04-23T08:19:48.419Z
617ba1820e4237bd1731b867
/avatars/f9de06363e64bddd7dc977e96e85df8a.svg
zhengcong fei
user
onion
66271377b24310dd5f0c3a90
Mingyuan Fan
admin_assigned
2024-04-23T08:19:58.669Z
634672bfb7b4e71c7f45360f
/avatars/4b646fc3e271be90b9ec619d42ce3e99.svg
Fan Mingyuan
user
MichaelFan
66271377b24310dd5f0c3a91
Junshi Huang
admin_assigned
2024-04-23T08:20:11.771Z
642e8321be01b88c9447b25e
/avatars/cc760d9b1bd943da1e6d21f14cc6d495.svg
HuangJunshi
user
huangjunshi
Music Consistency Models
Consistency models have exhibited remarkable capabilities in facilitating
efficient image/video generation, enabling synthesis with minimal sampling
steps. It has proven to be advantageous in mitigating the computational burdens
associated with diffusion models. Nevertheless, the application of consistency
models in music generation remains largely unexplored. To address this gap, we
present Music Consistency Models (MusicCM), which leverages the
concept of consistency models to efficiently synthesize mel-spectrogram for
music clips, maintaining high quality while minimizing the number of sampling
steps. Building upon existing text-to-music diffusion models, the
MusicCM model incorporates consistency distillation and adversarial
discriminator training. Moreover, we find it beneficial to generate extended
coherent music by incorporating multiple diffusion processes with shared
constraints. Experimental results reveal the effectiveness of our model in
terms of computational efficiency, fidelity, and naturalness. Notable,
MusicCM achieves seamless music synthesis with a mere four sampling
steps, e.g., only one second per minute of the music clip, showcasing the
potential for real-time application.
Music Consistency Models (MusicCM) efficiently synthesize high-quality music by leveraging consistency models, reducing computational requirements and enabling real-time application.
consistency models Music Consistency Models mel-spectrogram diffusion models consistency distillation adversarial discriminator real-time application