Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias
Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training
verifiers to solve math word problems. arXiv:abs/2110.14168, 2021.

Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G. Carbonell, Quoc Viet Le, and Ruslan Salakhutdinov.
Transformer-XL: Attentive language models beyond a fixed-length context. In ACL (1), pp. 2978–2988.
Association for Computational Linguistics, 2019.

Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and
Rahul Gupta. BOLD: Dataset and metrics for measuring biases in open-ended language generation. In
FAccT, pp. 862–872. ACM, 2021.

Jiayu Ding, Shuming Ma, Li Dong, Xingxing Zhang, Shaohan Huang, Wenhui Wang, Nanning Zheng, and

Furu Wei. LongNet: Scaling transformers to 1, 000, 000, 000 tokens. arXiv:abs/2307.02486, 2023.
CodeLlama2