Memory is defined by two key components: input and output. Input: The input is information that is the same or similar to what the brain (or an LLM) has previously encountered (this is a necessary condition for memory—without input, there is no memory). Output: The result based on the input, which could be correct, incorrect, or forgotten. If the result is correct, it means it aligns with information previously acquired. (Note: for there to be an output, there must be an input; memory doesn't emerge spontaneously. Without specific input conditions, a person wouldn't recall a particular event. ) Using Question 1 as an example, the input is: “What is Newton's first law?” Without this input, no one would suddenly recall Newton's first law. The recollection of Newton's first law is triggered by input related to the theoretical context. This is why input is a necessary condition for memory, as it is the input that stimulates recall. The memory might be accurate, or it might be incorrect, indicating a deviation from previously acquired information—this deviation could be minor, significant, or even total forgetting. For example: Question 1: What is Newton's first law? Answer 2: Minor distortions Every object perseveres in its state of rest, except insofar as it is compelled to change that state by forces impressed thereon. Answer 3: Severe distortions Every object always perseveres in its state of rest. Answer 4: Memory lapse I do not know. In summary, the term ”memory” was traditionally used to refer specifically to human memory before the emergence of LLMs. Now, we believe that LLMs also exhibit memory. Therefore, we will verify the memory characteristics of LLMs based on the definition of memory outlined above. 3. 2 T HEMEMORY MECHANISM AND ABILITY OF LLM S In Section 2. 2, we have introduced the UAT format corresponding to Transformer-based LLMs. This UAT format can dynamically adjust to fit the corresponding output based on the input. Following this idea, could we also consider that the memory feature of LLMs adjusts to specific outputs based on the input? To explore this hypothesis, we designed a simple experiment. In the process of learning, we often need to memorize certain content, such as poems. For this ex-periment, we utilized publicly available datasets from Hugging Face: CN Poems (larryvrh/Chinese-Poems) for Chinese poetry and ENG Poems (jnb666/poems) for English poetry. We preprocessed the data in line with typical human memorization habits, allowing the LLMs to output the content of poems based on basic input information. For CN Poems, the input consisted of the dynasty, author, and title, while for ENG Poems, the input was the author and title. We restricted the length of input data to a maximum of 256 characters. Due to differences in char-acter encoding between Chinese and English, a single Chinese character usually corresponds to one token, while an English word may map to multiple tokens. As a result, after tokenization, the length of Chinese input remains almost unchanged, with a maximum of 256 tokens. In contrast, the En-glish input expands to a maximum of 730 tokens after tokenization. For the experiment, we selected 2,000 poems from each dataset. Table 1: The memory ability of Qwen1. 5-0. 5B-Chat, Qwen2-0. 5B-Instruct, Qwen2-1. 5B-Instruct, bloom-389m-zh, bloom-1b4-zh, bloom-560m, bloom-1b7 on CN Poems and ENG Poems. Models Qwen1. 5-0. 5B-Chat Qwen2-0. 5B-Instruct Qwen2-1. 5B-Instructbloom-389m-zhbloom-1b4-zhbloom-560mbloom-1b7 CN Poems Acc 68. 85 77. 5 96. 9 75. 55 96. 6--ENG Poems Acc 99. 85 99. 85 99. 9--99. 2 99. 15 To test the memory ability of LLMs, we fine-tuned the CN Poems and ENG Poems on Qwen series models Bai et al. (2023); Yang et al. (2024a) and bloom Workshop et al. (2023) models. The results in Table 1 indicate that LLMs possess memory capabilities and align perfectly with the definition 4