Token-based Audio Inpainting via Discrete Diffusion
Audio inpainting refers to the task of reconstructing missing segments in
corrupted audio recordings. While prior approaches-including waveform and
spectrogram-based diffusion models-have shown promising results for short gaps,
they often degrade in quality when gaps exceed 100 milliseconds (ms). In this
work, we introduce a novel inpainting method based on discrete diffusion
modeling, which operates over tokenized audio representations produced by a
pre-trained audio tokenizer. Our approach models the generative process
directly in the discrete latent space, enabling stable and semantically
coherent reconstruction of missing audio. We evaluate the method on the
MusicNet dataset using both objective and perceptual metrics across gap
durations up to 300 ms. We further evaluated our approach on the MTG dataset,
extending the gap duration to 500 ms. Experimental results demonstrate that our
method achieves competitive or superior performance compared to existing
baselines, particularly for longer gaps, offering a robust solution for
restoring degraded musical recordings. Audio examples of our proposed method
can be found at https://iftach21.github.io/
687796c5ff8f47a7f86442e9
Tali Dror
claimed_verified
2025-07-16T15:05:44.585Z
66f8662bce69a58cb5d7ccb3
/avatars/1ecccb3636bb749e169c7dda533a6d51.svg
Tali Dror
user
TaliDror
687796c5ff8f47a7f86442ea
Iftach Shoham
687796c5ff8f47a7f86442eb
Moshe Buchris
687796c5ff8f47a7f86442ec
Oren Gal
687796c5ff8f47a7f86442ed
Haim Permuter
687796c5ff8f47a7f86442ee
Gilad Katz
687796c5ff8f47a7f86442ef
Eliya Nachmani
Token-based Audio Inpainting via Discrete Diffusion
Audio inpainting refers to the task of reconstructing missing segments in
corrupted audio recordings. While prior approaches-including waveform and
spectrogram-based diffusion models-have shown promising results for short gaps,
they often degrade in quality when gaps exceed 100 milliseconds (ms). In this
work, we introduce a novel inpainting method based on discrete diffusion
modeling, which operates over tokenized audio representations produced by a
pre-trained audio tokenizer. Our approach models the generative process
directly in the discrete latent space, enabling stable and semantically
coherent reconstruction of missing audio. We evaluate the method on the
MusicNet dataset using both objective and perceptual metrics across gap
durations up to 300 ms. We further evaluated our approach on the MTG dataset,
extending the gap duration to 500 ms. Experimental results demonstrate that our
method achieves competitive or superior performance compared to existing
baselines, particularly for longer gaps, offering a robust solution for
restoring degraded musical recordings. Audio examples of our proposed method
can be found at https://iftach21.github.io/
A discrete diffusion model for audio inpainting using tokenized audio representations achieves competitive performance for reconstructing long gaps in corrupted audio recordings.
discrete diffusion modeling tokenized audio representations pre-trained audio tokenizer latent space MusicNet dataset MTG dataset audio inpainting generative process semantically coherent reconstruction