Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H.,
Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano,
R., Hesse, C., and Schulman, J. Training verifiers to solve
math word problems. arXiv preprint arXiv:2110.14168,
2021.

Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q. V., and
Salakhutdinov, R. Transformer-xl: Attentive language
models beyond a fixed-length context. arXiv preprint
arXiv:1901.02860, 2019.

Dao, T., Fu, D., Ermon, S., Rudra, A., and R´e, C. Flashat-
tention: Fast and memory-efficient exact attention with
io-awareness. Advances in Neural Information Process-
ing Systems, 35:16344–16359, 2022.

Ding, J., Ma, S., Dong, L., Zhang, X., Huang, S., Wang, W.,
Zheng, N., and Wei, F. Longnet: Scaling transformers to
1,000,000,000 tokens. arXiv preprint arXiv:2307.02486,
2023.
Self-Extend LLM