Frequency domain reconstruction loss: while the mel-reconstruction loss [18] is known to improve
stability, fidelity and convergence speed, the multi-scale spectral losses[42, 11, 15] encourage model-
ing of frequencies in multiple time-scales. In our model, we combine both methods by using a L1
loss on mel-spectrograms computed with window lengths of [32, 64, 128, 256, 512, 1024, 2048] and
hop length set to window_length / 4. We especially find that using the lowest hop size of 8 improves
modeling of very quick transients that are especially common in the music domain.
EnCodec [8] uses a similar loss formulation, but with both L1 and L2 loss terms, and a fixed mel bin
size of 64. We find that fixing mel bin size leads to holes in the spectrogram especially at low filter
lengths. Therefore, we use mel bin sizes [5, 10, 20, 40, 80, 160, 320] corresponding to the above filter
lengths which were verified to be correct by manual inspection.
RVQGAN