Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang
Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang,
Yuandong Tian, Christopher Re, et al. 2023b. Deja
vu: Contextual sparsity for efficient llms at infer-
ence time. In International Conference on Machine
Learning, pages 22137–22176. PMLR.

Moinuddin K Meswani, Sergey Blagodurov, David
Roberts, John Slice, Mike Ignatowski, and Gabriel
Loh. 2015. Neural cache: Bit-serial in-cache acceler-
ation of deep neural networks. In 2015 48th Annual
IEEE/ACM International Symposium on Microarchi-
tecture (MICRO), pages 383–394. IEEE.

Iman Mirzadeh, Keivan Alizadeh, Sachin Mehta, Carlo
C Del Mundo, Oncel Tuzel, Golnoosh Samei, Mo-
hammad Rastegari, and Mehrdad Farajtabar. 2023.
Relu strikes back: Exploiting activation sparsity in
large language models.
LLM in a flash