designed to operate at a lower frame rate (50 Hz) than DAC (86 Hz), thus reducing by 40% the
inference runtime for producing the same audio. Note finally that DAC was trained on a different
dataset than EnCodec, and further experiments would be required to understand exactly what
influences the fitness of such compression models to be used with auto-regressive language models.
Effect of D-Adaptation.
D-Adaptation is a novel automated way of selecting the overall learning rate of the Adam optimizer,
i.e. its α parameter, dynamically throughout the training, introduced by Defazio and Mishchenko
[2023]. We observed improved convergence for the 300M parameter model, although for larger
models, e.g. 1.5B and 3.3B, we observed the automated rule led to deteriorated performance, both
on the train and validation set. Further investigation is required to better understand the effect of
D-Adaptation, and whether it can scale to the largest model. The convergence for both methods can
Simple and Controllable Music Generation