EDMSound: Spectrogram Based Diffusion Models for Efficient and
  High-Quality Audio Synthesis
Audio diffusion models can synthesize a wide variety of sounds. Existing
models often operate on the latent domain with cascaded phase recovery modules
to reconstruct waveform. This poses challenges when generating high-fidelity
audio. In this paper, we propose EDMSound, a diffusion-based generative model
in spectrogram domain under the framework of elucidated diffusion models (EDM).
Combining with efficient deterministic sampler, we achieved similar Fr\'echet
audio distance (FAD) score as top-ranked baseline with only 10 steps and
reached state-of-the-art performance with 50 steps on the DCASE2023 foley sound
generation benchmark. We also revealed a potential concern regarding diffusion
based audio generation models that they tend to generate samples with high
perceptual similarity to the data from training data. Project page:
https://agentcooper2002.github.io/EDMSound/
6555b2959dc61e22c51e2c09
Ge Zhu
claimed_verified
2023-11-16T15:51:14.761Z
6526f16b00040a5395b82c96
/avatars/592366207bc919a6984b3a0d816314c7.svg
Ge Zhu
user
gzhu06
6555b2959dc61e22c51e2c0a
Yutong Wen
6555b2959dc61e22c51e2c0b
Marc-André Carbonneau
6555b2959dc61e22c51e2c0c
Zhiyao Duan
claimed_verified
2023-11-17T07:41:41.267Z
6556f3c3ee35f7d8bcf7706d
/avatars/423682fbf354f085232237cbf0f51bbe.svg
Zhiyao Duan
user
zhiyaoduan
EDMSound: Spectrogram Based Diffusion Models for Efficient and
  High-Quality Audio Synthesis
Audio diffusion models can synthesize a wide variety of sounds. Existing
models often operate on the latent domain with cascaded phase recovery modules
to reconstruct waveform. This poses challenges when generating high-fidelity
audio. In this paper, we propose EDMSound, a diffusion-based generative model
in spectrogram domain under the framework of elucidated diffusion models (EDM).
Combining with efficient deterministic sampler, we achieved similar Fr\'echet
audio distance (FAD) score as top-ranked baseline with only 10 steps and
reached state-of-the-art performance with 50 steps on the DCASE2023 foley sound
generation benchmark. We also revealed a potential concern regarding diffusion
based audio generation models that they tend to generate samples with high
perceptual similarity to the data from training data. Project page:
https://agentcooper2002.github.io/EDMSound/
EDMSound, an audio diffusion model in the spectrogram domain using an efficient deterministic sampler, achieves state-of-the-art performance with fewer steps and highlights concerns about perceptual similarity in diffusion-based audio generation.
diffusion models latent domain cascaded phase recovery modules spectrogram domain elucidated diffusion models EDM efficient deterministic sampler Fréchet audio distance FAD DCASE2023 foley sound generation benchmark