  Timbre transfer techniques aim at converting the sound of a musical piece
generated by one instrument into the same one as if it was played by another
instrument, while maintaining as much as possible the content in terms of
musical characteristics such as melody and dynamics. Following their recent
breakthroughs in deep learning-based generation, we apply Denoising Diffusion
Models (DDMs) to perform timbre transfer. Specifically, we apply the recently
proposed Denoising Diffusion Implicit Models (DDIMs) that enable to accelerate
the sampling procedure. Inspired by the recent application of DDMs to image
translation problems we formulate the timbre transfer task similarly, by first
converting the audio tracks into log mel spectrograms and by conditioning the
generation of the desired timbre spectrogram through the input timbre
spectrogram. We perform both one-to-one and many-to-many timbre transfer, by
converting audio waveforms containing only single instruments and multiple
instruments, respectively. We compare the proposed technique with existing
state-of-the-art methods both through listening tests and objective measures in
order to demonstrate the effectiveness of the proposed model.

Audio and Speech Processing