Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion
Editing signals using large pre-trained models, in a zero-shot manner, has
recently seen rapid advancements in the image domain. However, this wave has
yet to reach the audio domain. In this paper, we explore two zero-shot editing
techniques for audio signals, which use DDPM inversion on pre-trained diffusion
models. The first, adopted from the image domain, allows text-based editing.
The second, is a novel approach for discovering semantically meaningful editing
directions without supervision. When applied to music signals, this method
exposes a range of musically interesting modifications, from controlling the
participation of specific instruments to improvisations on the melody. Samples
can be found on our examples page in https://hilamanor.github.io/AudioEditing/
and code can be found in https://github.com/hilamanor/AudioEditing/ .
65ceee613db7d7b915d25236
Hila Manor
claimed_verified
2024-02-16T10:26:58.615Z
63f6454a9cbd673030295df4
/avatars/90c0e4a1c5dffdabbf2492853741d0d9.svg
Hila Manor
user
hilamanor
65ceee613db7d7b915d25237
Tomer Michaeli
admin_assigned
2024-02-16T11:10:59.321Z
6457c3f50664b05a5b1d941d
/avatars/0132be4d068d4b314f6b498b65411461.svg
Tomer Michaeli
user
Michato
Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion
Editing signals using large pre-trained models, in a zero-shot manner, has
recently seen rapid advancements in the image domain. However, this wave has
yet to reach the audio domain. In this paper, we explore two zero-shot editing
techniques for audio signals, which use DDPM inversion on pre-trained diffusion
models. The first, adopted from the image domain, allows text-based editing.
The second, is a novel approach for discovering semantically meaningful editing
directions without supervision. When applied to music signals, this method
exposes a range of musically interesting modifications, from controlling the
participation of specific instruments to improvisations on the melody. Samples
can be found on our examples page in https://hilamanor.github.io/AudioEditing/
and code can be found in https://github.com/hilamanor/AudioEditing/ .
Two zero-shot techniques using DDPM inversion on pre-trained diffusion models are explored for editing audio signals, enabling text-based manipulation and discovery of semantically meaningful editing directions.
DDPM inversion diffusion models zero-shot editing text-based editing semantically meaningful editing directions music signals instrument control melody improvisation