title: MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity
    Reconstruction and Generation
summary: Neural audio codecs have made significant strides in efficiently mapping raw
  audio waveforms into discrete token representations, which are foundational for
  contemporary audio generative models. However, most existing codecs are
  optimized primarily for reconstruction quality, often at the expense of the
  downstream modelability of the encoded tokens. Motivated by the need to
  overcome this bottleneck, we introduce MagiCodec, a novel
  single-layer, streaming Transformer-based audio codec. MagiCodec is designed
  with a multistage training pipeline that incorporates Gaussian noise injection
  and latent regularization, explicitly targeting the enhancement of semantic
  expressiveness in the generated codes while preserving high reconstruction
  fidelity. We analytically derive the effect of noise injection in the frequency
  domain, demonstrating its efficacy in attenuating high-frequency components and
  fostering robust tokenization. Extensive experimental evaluations show that
  MagiCodec surpasses state-of-the-art codecs in both reconstruction quality and
  downstream tasks. Notably, the tokens produced by MagiCodec exhibit Zipf-like
  distributions, as observed in natural languages, thereby improving
  compatibility with language-model-based generative architectures. The code and
  pre-trained models are available at https://github.com/Ereboas/MagiCodec.
paper_authors._id: 683e707763e27c6256f58a51
paper_authors.name: Yakun Song
paper_authors._id: 683e707763e27c6256f58a52
paper_authors.name: Jiawei Chen
paper_authors._id: 683e707763e27c6256f58a53
paper_authors.name: Xiaobin Zhuang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-03T08:45:12.616Z
paper_authors._id: 63774ca43a63a2983ffc12f9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63774ca43a63a2983ffc12f9/jUJasGf5_2rwiEun_eYl7.png
paper_authors.fullname: xiaobin zhuang
paper_authors.type: user
paper_authors.user: xiaobinzhuang
paper_authors._id: 683e707763e27c6256f58a54
paper_authors.name: Chenpeng Du
paper_authors._id: 683e707763e27c6256f58a55
paper_authors.name: Ziyang Ma
paper_authors._id: 683e707763e27c6256f58a56
paper_authors.name: Jian Wu
paper_authors._id: 683e707763e27c6256f58a57
paper_authors.name: Jian Cong
paper_authors._id: 683e707763e27c6256f58a58
paper_authors.name: Dongya Jia
paper_authors._id: 683e707763e27c6256f58a59
paper_authors.name: Zhuo Chen
paper_authors._id: 683e707763e27c6256f58a5a
paper_authors.name: Yuping Wang
paper_authors._id: 683e707763e27c6256f58a5b
paper_authors.name: Yuxuan Wang
paper_authors._id: 683e707763e27c6256f58a5c
paper_authors.name: Xie Chen
paper_title: MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity
    Reconstruction and Generation
paper_summary: Neural audio codecs have made significant strides in efficiently mapping raw
  audio waveforms into discrete token representations, which are foundational for
  contemporary audio generative models. However, most existing codecs are
  optimized primarily for reconstruction quality, often at the expense of the
  downstream modelability of the encoded tokens. Motivated by the need to
  overcome this bottleneck, we introduce MagiCodec, a novel
  single-layer, streaming Transformer-based audio codec. MagiCodec is designed
  with a multistage training pipeline that incorporates Gaussian noise injection
  and latent regularization, explicitly targeting the enhancement of semantic
  expressiveness in the generated codes while preserving high reconstruction
  fidelity. We analytically derive the effect of noise injection in the frequency
  domain, demonstrating its efficacy in attenuating high-frequency components and
  fostering robust tokenization. Extensive experimental evaluations show that
  MagiCodec surpasses state-of-the-art codecs in both reconstruction quality and
  downstream tasks. Notably, the tokens produced by MagiCodec exhibit Zipf-like
  distributions, as observed in natural languages, thereby improving
  compatibility with language-model-based generative architectures. The code and
  pre-trained models are available at https://github.com/Ereboas/MagiCodec.
paper_ai_summary: MagiCodec, a Transformer-based audio codec, enhances semantic tokenization while maintaining high reconstruction quality, improving compatibility with generative models.
paper_ai_keywords: Transformer Gaussian noise injection latent regularization frequency domain Zipf-like distributions generative models