Since then, many LLMs have emerged, including
LAMBDA (Thoppilan et al., 2022), PaLM (Chowd-
hery et al., 2022), OPT (Zhang et al., 2022a),
LLaMA (Touvron et al., 2023), BLOOM (Work-
shop et al., 2023), Galactica (Taylor et al., 2022),
and Pangu (Zeng et al., 2021; Ren et al., 2023). One
of the most notable works in this series of research
that has attracted widespread industry attention and
can be considered a milestone towards Artiﬁcial
General Intelligence (AGI) is ChatGPT (OpenAI,
2022), which is based on InsctuctGPT (Ouyang
et al., 2022) and optimized for multi-turn dialogue
ability. ChatGPT has achieved remarkable perfor-
mance and surpassed the boundaries between NLP
tasks. However, current LLMs, including ChatGPT,
face signiﬁcant limitations when processing tasks
involving extremely long inputs.
Unleashing Infinite-Length Input Capacity for Large-scale Language Models with Self-Controlled Memory System