of multi-layer Transformers aligns with the UAT in a general sense. However, unlike the original UAT, the UAT form of Transformer-based models has the ability to dynamically fit functions based on the input. Figure 1 illustrates a basic Transformer module, and according to UAT2LLMs, the corresponding UAT form for Figure 1 is: xi`1“p W1 i`1,1x0`bi`1,1q`i`1ÿ j“1W1 j,3σp W1 j,2x1 0`b1 j,2q (3) where xi`1is the output of i`1-th layer, x0is the input of the network, b1 j,2“p W1 j,2b1 j´1,3` b1 j,2q`W1 j,2UATR j´1, where UATR j´1“řj´1 k“1W1 k,3σp W1 k,2x1 0`b1 k,2q. The term b1 j,2is approx-imated by the jlayer of UAT with x0as input. This enhances the model's ability to dynamically adjust functions based on input. In the multi-head attention mechanism, the parameters change dy-namically with the input. Therefore, in the formula above, all W1 j,1,W1 j,2, and W1 j,3parameters in layer i, where j“1,..., i, are dynamically adjusted based on the input. 3 M EMORY In this section, we will demonstrate the memory capabilities of LLMs. First, in Section 3. 1, we provide a clear definition of memory. Then, in Section 3. 2, we explain the memory mechanism of LLMs using UAT theory and validate their memory characteristics through experiments. In Section 3. 3, we explore the impact of input length on the accuracy of LLM memory. 3. 1 T HEDEFINITION OF MEMORY Before formally studying memory in LLMs, it's important to first define or provide a relatively accurate description of memory. According to Wikipedia: ”Memory is the faculty of the mind by which data or information is encoded, stored, and retrieved when needed. ” However, this definition has some fundamental issues. Encoding data or information is not prob-lematic, as information in the brain is transmitted via electrical signals, and we need to encode that information in a way the brain can process. The problem arises with the concepts of ”storage” and ”retrieval. ” The brain does not have a structure analogous to a database for storing information. So, where is this information actually stored? Is it in the neurons of the brain? If so, does a single neuron store a word, or does it store an entire sentence? Question 1: What is Newton's first law? Answer 1:Every object perseveres in its state of rest, or of uniform motion in a right line,except insofar as it is compelled to change that state by forces impressed thereon. So, is this sentence stored within a single neuron? Or does each neuron store just a word, with a spe-cific region of the brain dedicated to this particular memory? Given the vast amount of information humans receive daily, can neurons truly store such an immense volume of data without hindering normal cognitive processes? After all, almost every routine activity requires memory. Take, for example, the simple task of going to the cafeteria: we need to remember when to go, the cafeteria's location, the route to get there, which foods are available, what counts as utensils, where to find them, and how to use them. Moreover, if this memory is stored in a fixed set of neurons, then every time the question is raised, the response should be identical, since the retrieval would be from the same static content. Every word in the response should be exact, with no omissions or additions (even if the information has been abstractly encoded, as long as the encoding and decoding processes are consistent, the content should remain unchanged). This, however, is clearly unreasonable. Therefore, we need a more precise definition of the concept of ”memory”: 3