  This paper presents a novel high-fidelity and low-latency universal neural
vocoder framework based on multiband WaveRNN with data-driven linear prediction
for discrete waveform modeling (MWDLP). MWDLP employs a coarse-fine bit WaveRNN
architecture for 10-bit mu-law waveform modeling. A sparse gated recurrent unit
with a relatively large size of hidden units is utilized, while the multiband
modeling is deployed to achieve real-time low-latency usage. A novel technique
for data-driven linear prediction (LP) with discrete waveform modeling is
proposed, where the LP coefficients are estimated in a data-driven manner.
Moreover, a novel loss function using short-time Fourier transform (STFT) for
discrete waveform modeling with Gumbel approximation is also proposed. The
experimental results demonstrate that the proposed MWDLP framework generates
high-fidelity synthetic speech for seen and unseen speakers and/or language on
300 speakers training data including clean and noisy/reverberant conditions,
where the number of training utterances is limited to 60 per speaker, while
allowing for real-time low-latency processing using a single core of $\sim\!$
2.1--2.7 GHz CPU with $\sim\!$ 0.57--0.64 real-time factor including
input/output and feature extraction.

Sound;Computation and Language;Machine Learning;Audio and Speech Processing