SCHRODINGER 'SMEMORY : LARGE LANGUAGE MOD-ELS Wei Wang Department of Computing The Hong Kong Polytechnic University weiuat. wang@connect. polyu. hk Qing Li Department of Computing The Hong Kong Polytechnic University qing-prof. li@polyu. edu. hk ABSTRACT Memory is the foundation of LLMs' functionality, yet past research has lacked an in-depth exploration of their memory capabilities and underlying theory. In this paper, we apply UAT theory to explain the memory mechanism of LLMs and pro-pose a new approach for evaluating LLM performance by comparing the memory capacities of different models. Through extensive experiments, we validate our theory and the memory abilities of LLMs. Finally, we compare the capabilities of the human brain and LLMs, highlighting both their similarities and differences in terms of working mechanisms. 1 I NTRODUCTION Language is not only one of humanity's most essential capabilities but also the foundation of com-munication, knowledge transfer, and the development of civilization. Language models can be seen as a simulation of human intelligence, enabling them to perform tasks that traditionally required human input. Large Language Models (LLMs), particularly those based on the Transformer archi-tecture, are among the hottest research topics in artificial intelligence today. They have begun to mimic human language abilities and are already impacting various aspects of everyday life, such as machine translation, text summarization, sentiment analysis, question-answering systems, and text generation. Despite the impressive capabilities of LLMs, the research on their memory mechanisms remains underexplored. Memory is a crucial ability for humans; without it, we would struggle to complete even the simplest tasks. For instance, something as routine as eating requires remembering where to eat, how to get there, what to eat, and which utensils to use. These everyday memories have become second nature to us, so much so that we rarely think about them consciously. Clearly, memory plays a vital role in guiding nearly every aspect of our daily lives. As LLMs become more powerful, it raises the question: do these models possess memory? If so, what form does it take? How does it differ from human memory? Current research into LLM memory mechanisms primarily follows two directions: Expanding Context Length: This approach aims to equip LLMs with long-term memory by extend-ing the context window Chen et al. (2023); Zhu et al. (2023); Yang (2023); Fei et al. (2023). Since short contexts fail to provide enough information, increasing the context length allows the model to maintain more comprehensive information across long sequences. External Memory Integration: This method involves building memory storage systems Graves et al. (2014); Xiao et al. (2024); Wu et al. (2022); Yang et al. (2024b) that encode and store past events Zhang et al. (2023), allowing the model to retrieve and update memories as needed. Such mecha-nisms enable models to forget or reinforce certain memories over time. While these studies have made strides in addressing the memory limitations of LLMs, they do not fully explain how memory works within these models. For example, when asked ”Who is the Presi-dent of the United States?”, LLMs like GPT-4 or Llama-3 might answer with outdated information, indicating that some form of memory is present. However, this memory isn't from an external stor-age unit but is inferred from the model's training data. This raises fundamental questions: Why do LLMs exhibit this inferential memory? How does it compare to human memory? 1ar Xiv:2409. 10482v1 [cs. CL] 16 Sep 2024