arXiv:1607.06450, 2016.

Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. FlashAttention: Fast and
memory-efficient exact attention with IO-awareness. In Advances in Neural Information Processing
Systems, 2022.

Benjamin Lefaudeux, Francisco Massa, Diana Liskovich, Wenhan Xiong, Vittorio Caggiano, Sean
Naren, Min Xu, Jieru Hu, Marta Tintore, Susan Zhang, Patrick Labatut, and Daniel Haziza.
xformers: A modular and hackable transformer modelling library. https://github.com/
facebookresearch/xformers, 2022.

Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.

arXiv preprint

arXiv:1711.05101, 2017.

Aaron Defazio and Konstantin Mishchenko. Learning-rate-free learning by d-adaptation. arXiv

preprint arXiv:2301.07733, 2023.

Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. arXiv preprint

arXiv:1805.04833, 2018.

https://riffusion. com/about.
Simple and Controllable Music Generation