Danny Hernandez, Tom Brown, Tom Conerly, Nova
DasSarma, Dawn Drain, Sheer El-Showk, Nelson
Elhage, Zac Hatfield-Dodds, Tom Henighan, Tristan
Hume, et al. 2022. Scaling laws and interpretabil-
ity of learning from repeated data. arXiv preprint
arXiv:2205.10487.

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch,
Elena Buchatskaya, Trevor Cai, Eliza Rutherford,
Diego de Las Casas, Lisa Anne Hendricks, Johannes
Welbl, Aidan Clark, et al. 2022. An empirical analy-
sis of compute-optimal large language model training.
Advances in Neural Information Processing Systems,
35:30016–30030.

Hamish Ivison, Noah A. Smith, Hannaneh Hajishirzi,
and Pradeep Dasigi. 2023. Data-efficient finetuning
using cross-task nearest neighbors. In Findings of
the Association for Computational Linguistics: ACL
2023, pages 9036–9061, Toronto, Canada. Associa-
tion for Computational Linguistics.
DataManagementForLargeLanguageModels-ASurvey