Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen,

and Luke Zettlemoyer. Detecting pretraining data from large language models. CoRR, 2023.

Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew
Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning. arXiv
preprint arXiv:2010.03768, 2020.

Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared
Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, et al. Using deepspeed
and megatron to train megatron-turing nlg 530b, a large-scale generative language model. arXiv
preprint arXiv:2201.11990, 2022.

Qiaoyu Tang, Ziliang Deng, Hongyu Lin, Xianpei Han, Qiao Liang, and Le Sun. Toolalpaca: General-
ized tool learning for language models with 3000 simulated cases. arXiv preprint arXiv:2306.05301,
2023a.

22
ChatGPT’sOne-yearAnniversary-AreOpen-Source LargeLanguageModelsCatchingup