title: MusicHiFi: Fast High-Fidelity Stereo Vocoding
summary: Diffusion-based audio and music generation models commonly generate music by
  constructing an image representation of audio (e.g., a mel-spectrogram) and
  then converting it to audio using a phase reconstruction model or vocoder.
  Typical vocoders, however, produce monophonic audio at lower resolutions (e.g.,
  16-24 kHz), which limits their effectiveness. We propose MusicHiFi -- an
  efficient high-fidelity stereophonic vocoder. Our method employs a cascade of
  three generative adversarial networks (GANs) that convert low-resolution
  mel-spectrograms to audio, upsamples to high-resolution audio via bandwidth
  expansion, and upmixes to stereophonic audio. Compared to previous work, we
  propose 1) a unified GAN-based generator and discriminator architecture and
  training procedure for each stage of our cascade, 2) a new fast, near
  downsampling-compatible bandwidth extension module, and 3) a new fast
  downmix-compatible mono-to-stereo upmixer that ensures the preservation of
  monophonic content in the output. We evaluate our approach using both objective
  and subjective listening tests and find our approach yields comparable or
  better audio quality, better spatialization control, and significantly faster
  inference speed compared to past work. Sound examples are at
  https://MusicHiFi.github.io/web/.
paper_authors._id: 65f7a0f8d64a62e375b1f4f3
paper_authors.name: Ge Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-18T09:11:02.008Z
paper_authors._id: 6526f16b00040a5395b82c96
paper_authors.avatarUrl: /avatars/592366207bc919a6984b3a0d816314c7.svg
paper_authors.fullname: Ge Zhu
paper_authors.type: user
paper_authors.user: gzhu06
paper_authors._id: 65f7a0f8d64a62e375b1f4f4
paper_authors.name: Juan-Pablo Caceres
paper_authors._id: 65f7a0f8d64a62e375b1f4f5
paper_authors.name: Zhiyao Duan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-18T09:11:26.168Z
paper_authors._id: 6556f3c3ee35f7d8bcf7706d
paper_authors.avatarUrl: /avatars/423682fbf354f085232237cbf0f51bbe.svg
paper_authors.fullname: Zhiyao Duan
paper_authors.type: user
paper_authors.user: zhiyaoduan
paper_authors._id: 65f7a0f8d64a62e375b1f4f6
paper_authors.name: Nicholas J. Bryan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-18T09:11:32.465Z
paper_authors._id: 648119ee7a741c7f33f49f25
paper_authors.avatarUrl: /avatars/183d2143ba20e1cc8712c63c055aadd7.svg
paper_authors.fullname: Nicholas J. Bryan
paper_authors.type: user
paper_authors.user: Njb
paper_title: MusicHiFi: Fast High-Fidelity Stereo Vocoding
paper_summary: Diffusion-based audio and music generation models commonly generate music by
  constructing an image representation of audio (e.g., a mel-spectrogram) and
  then converting it to audio using a phase reconstruction model or vocoder.
  Typical vocoders, however, produce monophonic audio at lower resolutions (e.g.,
  16-24 kHz), which limits their effectiveness. We propose MusicHiFi -- an
  efficient high-fidelity stereophonic vocoder. Our method employs a cascade of
  three generative adversarial networks (GANs) that convert low-resolution
  mel-spectrograms to audio, upsamples to high-resolution audio via bandwidth
  expansion, and upmixes to stereophonic audio. Compared to previous work, we
  propose 1) a unified GAN-based generator and discriminator architecture and
  training procedure for each stage of our cascade, 2) a new fast, near
  downsampling-compatible bandwidth extension module, and 3) a new fast
  downmix-compatible mono-to-stereo upmixer that ensures the preservation of
  monophonic content in the output. We evaluate our approach using both objective
  and subjective listening tests and find our approach yields comparable or
  better audio quality, better spatialization control, and significantly faster
  inference speed compared to past work. Sound examples are at
  https://MusicHiFi.github.io/web/.
paper_ai_summary: MusicHiFi is an efficient high-fidelity stereophonic vocoder using a cascade of GANs to improve audio quality and spatialization control.
paper_ai_keywords: diffusion-based audio music generation models mel-spectrogram vocoder generative adversarial networks GANs bandwidth expansion mono-to-stereo upmixer subjective listening tests