3.2.3 Overall Model Architecture
We illustrate the detailed process in Figure 4. Con-
sistent with the previous stage, we use vvv-objective
diffusion and the 1D U-Net architecture. When con-
dition on the text embedding eee, we use the U-Net
configuration fθθθg (zzzσt; σt, eee) to generate the com-
pressed latent zzz = Eθθθe(mmmwww). Then, the generator
Gθθθg (eee, ϵϵϵ, s) applies DDIM sampling and calls the
U-Net s times to generate an approximate latent ˆzzz
from the text embedding eee and starting noise ϵϵϵ. The
final generation stack during inference to obtain a
waveform is

ˆwww = Dθθθd(Gθθθg (eee, ϵϵϵg, sg), ϵϵϵd, sd) .

(6)

Figure 4: The training scheme of our text-conditioned
latent diffusion (TCLD) generator. During the denoising
process, we provide the U-Net a feature vector (
) and
a text embedding (

).

fθθθd is the proposed 1D U-Net, called repeatedly
during decoding.
Moûsai