[47] Shen, Z., Zhang, M., Zhao, H., Yi, S., Li, H.: Eﬃcient attention: Attention with
linear complexities. In: Proceedings of the IEEE/CVF Winter Conference on
Applications of Computer Vision, pp. 3531–3539 (2021)

[48] Zhai, S., Talbott, W., Srivastava, N., Huang, C., Goh, H., Zhang, R., Susskind,

J.: An attention free transformer. arXiv preprint arXiv:2105.14103 (2021)

[49] Rabe, M.N., Staats, C.: Self-attention does not need o(n2) memory. arXiv

preprint arXiv:2112.05682 (2022)

[50] Jang, H., Kim, J., Jo, J.-E., Lee, J., Kim, J.: Mnnfast: A fast and scalable system
architecture for memory-augmented neural networks. In: Proceedings of the 46th
International Symposium on Computer Architecture, pp. 250–263 (2019)

[51] Zandieh, A., Han, I., Daliri, M., Karbasi, A.: Kdeformer: Accelerating trans-

formers via kernel density estimation. arXiv preprint arXiv:2302.02451 (2023)

[52] Ma, X., Zhou, C., Kong, X., He, J., Gui, L., Neubig, G., May, J., Zettlemoyer, L.:

43
Beyond Efficiency