B.2. Decoder and Discriminator

The input of our decoder is latent variables generated from the prior or posterior encoders, so the input channel size of the
decoder is 192. For the last convolutional layer of the decoder, we remove a bias parameter, as it causes unstable gradient
scales during mixed precision training.
For the discriminator, HiFi-GAN uses the multi-period discriminator containing ﬁve sub-discriminators with periods
[2, 3, 5, 7, 11] and the multi-scale discriminator containing three sub-discriminators. To improve training efﬁciency, we
leave only the ﬁrst sub-discriminator of the multi-scale discriminator that operates on raw waveforms and discard two
sub-discriminators operating on average-pooled waveforms. The resultant discriminator can be seen as the multi-period
discriminator with periods [1, 2, 3, 5, 7, 11].

(a) Training procedure

(b) Inference procedure

(c) Dilated and depth-wise separable convo-
lutional residual block
ConditionalVariationalAutoencoderwithAdversarialLearningfor End-to-EndText-to-Speech