Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Kr-
ishnan, R., and Song, D. Pretrained transformers im-
prove out-of-distribution robustness. In Proceedings of
the 58th Annual Meeting of the Association for Com-
putational Linguistics, pp. 2744–2751, Online, July
2020. Association for Computational Linguistics. doi:
10.18653/v1/2020.acl- main.244. URL https:
//aclanthology.org/2020.acl-main.244.

Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E.,
Cai, T., Rutherford, E., de las Casas, D., Hendricks, L. A.,
Welbl, J., Clark, A., Hennigan, T., Noland, E., Millican,
K., van den Driessche, G., Damoc, B., Guy, A., Osindero,
S., Simonyan, K., Elsen, E., Vinyals, O., Rae, J. W., and
Sifre, L. An empirical analysis of compute-optimal large
language model training.
In Oh, A. H., Agarwal, A.,
Belgrave, D., and Cho, K. (eds.), Advances in Neural
Information Processing Systems, 2022. URL https:
//openreview.net/forum?id=iBBcRUlOAPR.
Eight Things to Know about Large Language Models