Anthropic.

Introducing 100K Context Windows, 2023. URL https://www.anthropic.com/index/

100k-context-windows.

Jacob Austin, Augustus Odena, Maxwell I. Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen
Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. Program synthesis with large language
models. arXiv:abs/2108.07732, 2021.

Mohammad Bavarian, Heewoo Jun, Nikolas Tezak, John Schulman, Christine McLeavey, Jerry Tworek, and

Mark Chen. Efficient training of language models to fill in the middle. arXiv:abs/2207.14255, 2022.

Iz Beltagy, Matthew E. Peters, and Arman Cohan. Longformer: The long-document transformer.

arXiv:abs/2004.05150, 2020.

Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. GPT-Neo: Large scale autoregressive

language modeling with mesh-tensorflow, 2021. URL https://doi.org/10.5281/zenodo.5297715.
CodeLlama2