Shaobo Li, Xiaoguang Li, Lifeng Shang, Zhenhua Dong,
Cheng-Jie Sun, Bingquan Liu, Zhenzhou Ji, Xin
Jiang, and Qun Liu. 2022. How pre-trained language
models capture factual knowledge? a causal-inspired
analysis. In Findings of the Association for Compu-
tational Linguistics: ACL 2022, pages 1720–1732.

Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Luke
Zettlemoyer, Omer Levy, Jason Weston, and Mike
Lewis. 2023a. Self-alignment with instruction back-
translation. arXiv preprint arXiv:2308.06259.

Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie
Del Giorno, Suriya Gunasekar, and Yin Tat Lee.
2023b. Textbooks are all you need ii: phi-1.5 techni-
cal report. arXiv preprint arXiv:2309.05463.

Shihao Liang, Kunlun Zhu, Runchu Tian, Yujia Qin,
Huadong Wang, Xin Cong, Zhiyuan Liu, Xiaojiang
Liu, and Maosong Sun. 2023. Exploring format
consistency for instruction tuning. arXiv preprint
arXiv:2307.15504.
DataManagementForLargeLanguageModels-ASurvey