Acknowledgements

We’d like to thank Itay Sagron, Lailin Chen, Mah-
yar Najibi, Qichen Fu, Moin Nabi, Sachin Mehta,
Mohammad Samragh, Matt Johnson, Etai Zalts-
man, Lin Chang, Dominic Giampaolo, Taal Uliel,
Aftab Munshi for the valuable discussion.

References
Udit Agrawal, Rangharajan Venkatesan, Brucek
Khailany, Stephen W Keckler, and William J Dally.
2022. Atomlayer: minimizing dram data movement
for ultra-sparse models on gpus. In Proceedings of
the 27th ACM International Conference on Archi-
tectural Support for Programming Languages and
Operating Systems, pages 223–238.

Arash Ahmadian, Saurabh Dash, Hongyu Chen, Bharat
Venkitesh, Stephen Gou, Phil Blunsom, A. Ustun,
and Sara Hooker. 2023. Intriguing properties of quan-
tization at scale. ArXiv, abs/2305.19268.
LLM in a flash