OpenAI. 2022. Introducing chatgpt.

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida,
Carroll L. Wainwright, Pamela Mishkin, Chong
Zhang, Sandhini Agarwal, Katarina Slama, Alex
Ray, John Schulman, Jacob Hilton, Fraser Kelton,
Luke Miller, Maddie Simens, Amanda Askell, Pe-
ter Welinder, Paul Christiano, Jan Leike, and Ryan
Lowe. 2022. Training language models to follow in-
structions with human feedback.

Jason Phang, Yao Zhao, and Peter J. Liu. 2022. Inves-
tigating efﬁciently extending transformers for long
input summarization.

Oﬁr Press, Noah Smith, and Mike Lewis. 2022. Train
short, test long: Attention with linear biases enables
input length extrapolation. In International Confer-
ence on Learning Representations.

Alec Radford, Jeff Wu, Rewon Child, David Luan,
Dario Amodei, and Ilya Sutskever. 2018. Improving
language understanding with unsupervised learning.
Unleashing Infinite-Length Input Capacity for Large-scale Language Models with Self-Controlled Memory System