[43] Raﬀel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y.,
Li, W., Liu, P.J.: Exploring the limits of transfer learning with a uniﬁed text-to-
text transformer. The Journal of Machine Learning Research 21(1), 5485–5551
(2020)

[44] Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P.,
Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models
are few-shot learners. Advances in neural information processing systems 33,
1877–1901 (2020)

[45] Kitaev, N., Kaiser, L., Levskaya, A.: Reformer: The eﬃcient transformer.
In: International Conference on Learning Representations (2020). https://
openreview.net/forum?id=rkgNKkHtvB

[46] Katharopoulos, A., Vyas, A., Pappas, N., Fleuret, F.: Transformers are rnns:
Fast autoregressive transformers with linear attention. In: International Confer-
ence on Machine Learning, pp. 5156–5165 (2020). PMLR
Beyond Efficiency