title: Presto! Distilling Steps and Layers for Accelerating Music Generation
summary: Despite advances in diffusion-based text-to-music (TTM) methods, efficient,
  high-quality generation remains a challenge. We introduce Presto!, an approach
  to inference acceleration for score-based diffusion transformers via reducing
  both sampling steps and cost per step. To reduce steps, we develop a new
  score-based distribution matching distillation (DMD) method for the EDM-family
  of diffusion models, the first GAN-based distillation method for TTM. To reduce
  the cost per step, we develop a simple, but powerful improvement to a recent
  layer distillation method that improves learning via better preserving hidden
  state variance. Finally, we combine our step and layer distillation methods
  together for a dual-faceted approach. We evaluate our step and layer
  distillation methods independently and show each yield best-in-class
  performance. Our combined distillation method can generate high-quality outputs
  with improved diversity, accelerating our base model by 10-18x (230/435ms
  latency for 32 second mono/stereo 44.1kHz, 15x faster than comparable SOTA) --
  the fastest high-quality TTM to our knowledge. Sound examples can be found at
  https://presto-music.github.io/web/.
paper_authors._id: 6704aababd2bdd04b6adf4ae
paper_authors.name: Zachary Novack
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-10-08T03:59:43.629Z
paper_authors._id: 643060c6cb3fe707b24c53a2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/643060c6cb3fe707b24c53a2/MIoM9hrX0vV4XRyrm-4Kz.jpeg
paper_authors.fullname: Zachary Novack
paper_authors.type: user
paper_authors.user: ZacharyNovack
paper_authors._id: 6704aababd2bdd04b6adf4af
paper_authors.name: Ge Zhu
paper_authors._id: 6704aababd2bdd04b6adf4b0
paper_authors.name: Jonah Casebeer
paper_authors._id: 6704aababd2bdd04b6adf4b1
paper_authors.name: Julian McAuley
paper_authors._id: 6704aababd2bdd04b6adf4b2
paper_authors.name: Taylor Berg-Kirkpatrick
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-08T09:29:04.746Z
paper_authors._id: 66845caa4b898a41b7af775c
paper_authors.avatarUrl: /avatars/e205290cd372b6b464509de279f6642d.svg
paper_authors.fullname: Taylor Berg-Kirkpatrick
paper_authors.type: user
paper_authors.user: tberg12
paper_authors._id: 6704aababd2bdd04b6adf4b3
paper_authors.name: Nicholas J. Bryan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-08T07:19:15.590Z
paper_authors._id: 648119ee7a741c7f33f49f25
paper_authors.avatarUrl: /avatars/183d2143ba20e1cc8712c63c055aadd7.svg
paper_authors.fullname: Nicholas J. Bryan
paper_authors.type: user
paper_authors.user: Njb
paper_title: Presto! Distilling Steps and Layers for Accelerating Music Generation
paper_summary: Despite advances in diffusion-based text-to-music (TTM) methods, efficient,
  high-quality generation remains a challenge. We introduce Presto!, an approach
  to inference acceleration for score-based diffusion transformers via reducing
  both sampling steps and cost per step. To reduce steps, we develop a new
  score-based distribution matching distillation (DMD) method for the EDM-family
  of diffusion models, the first GAN-based distillation method for TTM. To reduce
  the cost per step, we develop a simple, but powerful improvement to a recent
  layer distillation method that improves learning via better preserving hidden
  state variance. Finally, we combine our step and layer distillation methods
  together for a dual-faceted approach. We evaluate our step and layer
  distillation methods independently and show each yield best-in-class
  performance. Our combined distillation method can generate high-quality outputs
  with improved diversity, accelerating our base model by 10-18x (230/435ms
  latency for 32 second mono/stereo 44.1kHz, 15x faster than comparable SOTA) --
  the fastest high-quality TTM to our knowledge. Sound examples can be found at
  https://presto-music.github.io/web/.
paper_ai_keywords: score-based diffusion transformers diffusion model score-based distribution matching distillation (DMD) EDM-family of diffusion GAN-based distillation text-to-music (TTM) hidden state variance layer distillation best-in-class performance high-quality outputs diversity latency 32 second mono/stereo 44.1kHz