McCandlish, S. Scaling laws and interpretability of learn-
ing from repeated data. Computing Research Reposi-
tory, 05 2022. doi: 10.48550/arXiv.2205.10487. URL
https://arxiv.org/abs/2205.10487v1. Ver-
sion 1.

Hirota, Y., Nakashima, Y., and Garcia, N. Quantifying soci-
etal bias ampliﬁcation in image captioning. In Proceed-
ings of the IEEE/CVF Conference on Computer Vision
and Pattern Recognition, pp. 13450–13459, 2022.

Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E.,
Cai, T., Rutherford, E., Casas, D. d. L., Hendricks, L. A.,
Welbl, J., Clark, A., et al. Training compute-optimal
large language models. arXiv preprint arXiv:2203.15556,
2022.

Hu, H., Salcic, Z., Sun, L., Dobbie, G., Yu, P. S., and Zhang,
X. Membership inference attacks on machine learning: A
survey. ACM Computing Surveys (CSUR), 54(11s):1–37,
2022.

Huang, J., Shao, H., and Chang, K. C.-C. Are large pre-
trained language models leaking your personal informa-
tion? arXiv preprint arXiv:2205.12628, 2022.
Pythia- A Suite for Analyzing Large Language Models Across Training and Scaling