5.3 Efficiency of Our Model
Efficiency is another highlight of our model, where
we only need an inference time similar to the audio
length on a consumer GPU, which is several min-
utes, while many other text-to-audio models take
many GPU hours (Dhariwal et al., 2020; Kreuk
et al., 2022), as in Table 2. Our model is very
friendly for research at university labs, as each
model can be trained on a single A100 GPU in 1
week of training using a batch size of 32. This is
equivalent to around 1M steps for both the diffusion
autoencoder and latent generator. For inference, as
an example, a novel audio source of ∼43s can be

synthesized in less than 50s using a consumer GPU
with a DDIM sampler and a high step count (100
generation steps and 100 decoding steps).

We also calculate the exact inference statistics for
our Moûsai vs. Riffusion models in Table 4, and
find that our model needs less than 1/5 the inference
time, and almost half of the inference memory than
Riffusion does.
Moûsai