Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan
Li, Max Ryabinin, Beidi Chen, Percy Liang, Christo-
pher Ré, Ion Stoica, and Ce Zhang. 2023. Flexgen:
High-throughput generative inference of large lan-
guage models with a single GPU. In International
Conference on Machine Learning, ICML 2023, 23-29
July 2023, Honolulu, Hawaii, USA, volume 202 of
Proceedings of Machine Learning Research, pages
31094–31116. PMLR.

Vedant Subramani, Marios Savvides, Li Ping, and Sha-
ran Narang. 2022. Adapt: Parameter adaptive token-
wise inference for vision transformers. In Proceed-
ings of the 55th Annual IEEE/ACM International
Symposium on Microarchitecture.

Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter.
2023. A simple and effective pruning approach for
large language models. ArXiv, abs/2306.11695.
LLM in a flash