title: PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform
    Generation
summary: Recently, universal waveform generation tasks have been investigated
  conditioned on various out-of-distribution scenarios. Although GAN-based
  methods have shown their strength in fast waveform generation, they are
  vulnerable to train-inference mismatch scenarios such as two-stage
  text-to-speech. Meanwhile, diffusion-based models have shown their powerful
  generative performance in other domains; however, they stay out of the
  limelight due to slow inference speed in waveform generation tasks. Above all,
  there is no generator architecture that can explicitly disentangle the natural
  periodic features of high-resolution waveform signals. In this paper, we
  propose PeriodWave, a novel universal waveform generation model. First, we
  introduce a period-aware flow matching estimator that can capture the periodic
  features of the waveform signal when estimating the vector fields.
  Additionally, we utilize a multi-period estimator that avoids overlaps to
  capture different periodic features of waveform signals. Although increasing
  the number of periods can improve the performance significantly, this requires
  more computational costs. To reduce this issue, we also propose a single
  period-conditional universal estimator that can feed-forward parallel by
  period-wise batch inference. Additionally, we utilize discrete wavelet
  transform to losslessly disentangle the frequency information of waveform
  signals for high-frequency modeling, and introduce FreeU to reduce the
  high-frequency noise for waveform generation. The experimental results
  demonstrated that our model outperforms the previous models both in
  Mel-spectrogram reconstruction and text-to-speech tasks. All source code will
  be available at https://github.com/sh-lee-prml/PeriodWave.
paper_authors._id: 66bd908ea474a917fc01749e
paper_authors.name: Sang-Hoon Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-17T05:38:44.244Z
paper_authors._id: 654a62c211a85feec88ef8f2
paper_authors.avatarUrl: /avatars/aeff2661e7f3cbc6b3cdf43c6e31ca9b.svg
paper_authors.fullname: Sang-Hoon Lee
paper_authors.type: user
paper_authors.user: LeeSangHoon
paper_authors._id: 66bd908ea474a917fc01749f
paper_authors.name: Ha-Yeong Choi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-16T08:19:44.422Z
paper_authors._id: 6401a0c14a5c92eccfe333c2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6401a0c14a5c92eccfe333c2/-fv3jekUxMNrE058phGpA.png
paper_authors.fullname: Ha-Yeong Choi
paper_authors.type: user
paper_authors.user: Ha0
paper_authors._id: 66bd908ea474a917fc0174a0
paper_authors.name: Seong-Whan Lee
paper_title: PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform
    Generation
paper_summary: Recently, universal waveform generation tasks have been investigated
  conditioned on various out-of-distribution scenarios. Although GAN-based
  methods have shown their strength in fast waveform generation, they are
  vulnerable to train-inference mismatch scenarios such as two-stage
  text-to-speech. Meanwhile, diffusion-based models have shown their powerful
  generative performance in other domains; however, they stay out of the
  limelight due to slow inference speed in waveform generation tasks. Above all,
  there is no generator architecture that can explicitly disentangle the natural
  periodic features of high-resolution waveform signals. In this paper, we
  propose PeriodWave, a novel universal waveform generation model. First, we
  introduce a period-aware flow matching estimator that can capture the periodic
  features of the waveform signal when estimating the vector fields.
  Additionally, we utilize a multi-period estimator that avoids overlaps to
  capture different periodic features of waveform signals. Although increasing
  the number of periods can improve the performance significantly, this requires
  more computational costs. To reduce this issue, we also propose a single
  period-conditional universal estimator that can feed-forward parallel by
  period-wise batch inference. Additionally, we utilize discrete wavelet
  transform to losslessly disentangle the frequency information of waveform
  signals for high-frequency modeling, and introduce FreeU to reduce the
  high-frequency noise for waveform generation. The experimental results
  demonstrated that our model outperforms the previous models both in
  Mel-spectrogram reconstruction and text-to-speech tasks. All source code will
  be available at https://github.com/sh-lee-prml/PeriodWave.
paper_ai_keywords: GAN-based methods diffusion-based models waveform generation train-inference mismatch text-to-speech period-aware flow matching estimator vector fields multi-period estimator period-wise batch inference discrete wavelet transform frequency information high-frequency modeling FreeU Mel-spectrogram reconstruction