PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform
  Generation
Recently, universal waveform generation tasks have been investigated
conditioned on various out-of-distribution scenarios. Although GAN-based
methods have shown their strength in fast waveform generation, they are
vulnerable to train-inference mismatch scenarios such as two-stage
text-to-speech. Meanwhile, diffusion-based models have shown their powerful
generative performance in other domains; however, they stay out of the
limelight due to slow inference speed in waveform generation tasks. Above all,
there is no generator architecture that can explicitly disentangle the natural
periodic features of high-resolution waveform signals. In this paper, we
propose PeriodWave, a novel universal waveform generation model. First, we
introduce a period-aware flow matching estimator that can capture the periodic
features of the waveform signal when estimating the vector fields.
Additionally, we utilize a multi-period estimator that avoids overlaps to
capture different periodic features of waveform signals. Although increasing
the number of periods can improve the performance significantly, this requires
more computational costs. To reduce this issue, we also propose a single
period-conditional universal estimator that can feed-forward parallel by
period-wise batch inference. Additionally, we utilize discrete wavelet
transform to losslessly disentangle the frequency information of waveform
signals for high-frequency modeling, and introduce FreeU to reduce the
high-frequency noise for waveform generation. The experimental results
demonstrated that our model outperforms the previous models both in
Mel-spectrogram reconstruction and text-to-speech tasks. All source code will
be available at https://github.com/sh-lee-prml/PeriodWave.
66bd908ea474a917fc01749e
Sang-Hoon Lee
claimed_verified
2024-08-17T05:38:44.244Z
654a62c211a85feec88ef8f2
/avatars/aeff2661e7f3cbc6b3cdf43c6e31ca9b.svg
Sang-Hoon Lee
user
LeeSangHoon
66bd908ea474a917fc01749f
Ha-Yeong Choi
claimed_verified
2024-08-16T08:19:44.422Z
6401a0c14a5c92eccfe333c2
https://cdn-avatars.huggingface.co/v1/production/uploads/6401a0c14a5c92eccfe333c2/-fv3jekUxMNrE058phGpA.png
Ha-Yeong Choi
user
Ha0
66bd908ea474a917fc0174a0
Seong-Whan Lee
PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform
  Generation
Recently, universal waveform generation tasks have been investigated
conditioned on various out-of-distribution scenarios. Although GAN-based
methods have shown their strength in fast waveform generation, they are
vulnerable to train-inference mismatch scenarios such as two-stage
text-to-speech. Meanwhile, diffusion-based models have shown their powerful
generative performance in other domains; however, they stay out of the
limelight due to slow inference speed in waveform generation tasks. Above all,
there is no generator architecture that can explicitly disentangle the natural
periodic features of high-resolution waveform signals. In this paper, we
propose PeriodWave, a novel universal waveform generation model. First, we
introduce a period-aware flow matching estimator that can capture the periodic
features of the waveform signal when estimating the vector fields.
Additionally, we utilize a multi-period estimator that avoids overlaps to
capture different periodic features of waveform signals. Although increasing
the number of periods can improve the performance significantly, this requires
more computational costs. To reduce this issue, we also propose a single
period-conditional universal estimator that can feed-forward parallel by
period-wise batch inference. Additionally, we utilize discrete wavelet
transform to losslessly disentangle the frequency information of waveform
signals for high-frequency modeling, and introduce FreeU to reduce the
high-frequency noise for waveform generation. The experimental results
demonstrated that our model outperforms the previous models both in
Mel-spectrogram reconstruction and text-to-speech tasks. All source code will
be available at https://github.com/sh-lee-prml/PeriodWave.
PeriodWave is a novel universal waveform generation model that uses a period-aware flow matching estimator, multi-period estimator, and discrete wavelet transform to improve performance in Mel-spectrogram reconstruction and text-to-speech tasks.
period-aware flow matching estimator multi-period estimator single period-conditional universal estimator discrete wavelet transform FreeU waveform generation Mel-spectrogram reconstruction text-to-speech