Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion
<|reference_start|>Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion: Editing signals using large pre-trained models, in a zero-shot manner, has recently seen rapid advancements in the image domain. However, this wave has yet to reach the audio domain. In this paper, we explore two zero-shot editing techniques for audio signals, which use DDPM inversion with pre-trained diffusion models. The first, which we coin ZEro-shot Text-based Audio (ZETA) editing, is adopted from the image domain. The second, named ZEro-shot UnSupervized (ZEUS) editing, is a novel approach for discovering semantically meaningful editing directions without supervision. When applied to music signals, this method exposes a range of musically interesting modifications, from controlling the participation of specific instruments to improvisations on the melody. Samples and code can be found in https://hilamanor.github.io/AudioEditing/ .<|reference_end|>
@article{manor2024zero-shot,
    title={Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion},
    author={Hila Manor and Tomer Michaeli},
    journal={arXiv preprint arXiv:2402.10009},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.10009},
    primaryClass={cs.SD cs.LG eess.AS}
}