title: High-Fidelity Audio Compression with Improved RVQGAN
summary: Language models have been successfully used to model natural signals, such as
  images, speech, and music. A key component of these models is a high quality
  neural compression model that can compress high-dimensional natural signals
  into lower dimensional discrete tokens. To that end, we introduce a
  high-fidelity universal neural audio compression algorithm that achieves ~90x
  compression of 44.1 KHz audio into tokens at just 8kbps bandwidth. We achieve
  this by combining advances in high-fidelity audio generation with better vector
  quantization techniques from the image domain, along with improved adversarial
  and reconstruction losses. We compress all domains (speech, environment, music,
  etc.) with a single universal model, making it widely applicable to generative
  modeling of all audio. We compare with competing audio compression algorithms,
  and find our method outperforms them significantly. We provide thorough
  ablations for every design choice, as well as open-source code and trained
  model weights. We hope our work can lay the foundation for the next generation
  of high-fidelity audio modeling.
paper_authors._id: 6487c6a66d93bc7eb47d1e97
paper_authors.name: Rithesh Kumar
paper_authors._id: 6487c6a66d93bc7eb47d1e98
paper_authors.name: Prem Seetharaman
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-14T12:20:02.445Z
paper_authors._id: 6009e97993286684ae332cdd
paper_authors.avatarUrl: /avatars/2653d0809cba9f01a69a314e5a946f45.svg
paper_authors.fullname: Prem Seetharaman
paper_authors.type: user
paper_authors.user: pseeth
paper_authors._id: 6487c6a66d93bc7eb47d1e99
paper_authors.name: Alejandro Luebs
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-14T12:20:19.592Z
paper_authors._id: 63f667554b831cc179c01c28
paper_authors.avatarUrl: /avatars/4c8029acacb96c5d70638a72ebdd5c01.svg
paper_authors.fullname: Alejandro Luebs
paper_authors.type: user
paper_authors.user: aluebs
paper_authors._id: 6487c6a66d93bc7eb47d1e9a
paper_authors.name: Ishaan Kumar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-14T14:54:16.220Z
paper_authors._id: 6303d3e17b50dd9d0a35b7ff
paper_authors.avatarUrl: /avatars/e9462ede328a7cac05d7d08f0a9961d4.svg
paper_authors.fullname: IshaanKumar
paper_authors.type: user
paper_authors.user: eeishaan
paper_authors._id: 6487c6a66d93bc7eb47d1e9b
paper_authors.name: Kundan Kumar
paper_title: High-Fidelity Audio Compression with Improved RVQGAN
paper_summary: Language models have been successfully used to model natural signals, such as
  images, speech, and music. A key component of these models is a high quality
  neural compression model that can compress high-dimensional natural signals
  into lower dimensional discrete tokens. To that end, we introduce a
  high-fidelity universal neural audio compression algorithm that achieves ~90x
  compression of 44.1 KHz audio into tokens at just 8kbps bandwidth. We achieve
  this by combining advances in high-fidelity audio generation with better vector
  quantization techniques from the image domain, along with improved adversarial
  and reconstruction losses. We compress all domains (speech, environment, music,
  etc.) with a single universal model, making it widely applicable to generative
  modeling of all audio. We compare with competing audio compression algorithms,
  and find our method outperforms them significantly. We provide thorough
  ablations for every design choice, as well as open-source code and trained
  model weights. We hope our work can lay the foundation for the next generation
  of high-fidelity audio modeling.
paper_ai_summary: A universal neural audio compression algorithm achieves high fidelity at 8 kbps bandwidth by combining advancements in audio generation, vector quantization, and improved loss functions.
paper_ai_keywords: neural compression model high-fidelity audio generation vector quantization adversarial loss reconstruction loss universal model