Lukasz Lew, Vlad Feinberg, Shivani Agrawal, Jihwan Lee, Jonathan Malmaud, Lisa Wang, Pouya
Dormiani, and Reiner Pope. Aqt: Accurate quantized training), 2022. URL http://github.
com/google/aqt.

Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning ﬁlters for

efﬁcient convnets. arXiv preprint arXiv:1608.08710, 2016.

Zhuohan Li, Eric Wallace, Sheng Shen, Kevin Lin, Kurt Keutzer, Dan Klein, and Joey Gonzalez.
Train big, then compress: Rethinking model size for efﬁcient training and inference of trans-
formers. In Proceedings of the 37th International Conference on Machine Learning, 2020. URL
https://proceedings.mlr.press/v119/li20m.html.

Shiwei Liu and Zhangyang Wang. Ten lessons we have learned in the new”sparseland”: A short

handbook for sparse neural network researchers. 2023.

10

Published as a conference paper at ICLR 2023
JAXPRUNER