This paper will leverage the Universal Approximation Theorem (UAT) to explain the reasoning and memory abilities of LLMs. The contributions of this work are as follows: An explanation of LLMs' memory and reasoning abilities through the lens of UAT. A new, more objective method for evaluating LLMs' memory and reasoning strengths. A comparison between the memory and reasoning capabilities of LLMs and human cogni-tion. The structure of this paper is as follows: In Section 2, we briefly explain UAT theory and present the mathematical form of UAT as it applies to multi-layer Transformers. In Section 3, we theoretically and experimentally demonstrate that LLMs possess memory capabilities. Finally, in Section 4, we conduct a comprehensive analysis of human and LLM abilities, focusing on memory capacity. 2 UAT AND LLM S The UAT (Cybenko, 2007; Popescu et al., 2009) serves as the foundational theory of deep learning. Our goal is to theoretically explain Transformer-based LLMs using the UAT framework. To do this, we will first present the mathematical form of UAT in Section 2. 1, followed by the correspond-ing UAT form for LLMs in Section 2. 2. We will then use this UAT form to explain the memory characteristics of LLMs. 2. 1 UAT In this section, we will briefly introduce the UAT. The UAT was originally introduced by Cybenko (2007). According to Theorem 2 from Cybenko (2007), if σis any continuous sigmoidal function, then a finite sum of the following form: Gpxq“Nÿ j“1αjσ` WT jx`θj˘ (1) is dense in Cp Inq. Here, Wj PRnandαj, θPRare fixed. For any f PCp Inqandεą0, there exists a function Gpxq: |Gpxq´fpxq|ăεfor all x PIn. (2) This indicates that, with a sufficiently large N, a neural network can approximate any continuous function over a closed interval. Hornik et al. (1989) further establishes that multilayer feedfor-ward networks adhere to the UAT, capable of approximating arbitrary Borel measurable functions. Considering Equation (1), where the function Gpxqoutputs a scalar in R, the framework naturally extends when Gpxqmaps to Rm, requiring approximation across each dimension. To handle this multidimensional output, a straightforward modification to Equation (1) is sufficient: the transfor-mation matrix Wjis adjusted to lie in the space Rnˆm, the bias term θjis redefined as a vector in Rm, andαjis reshaped into a matrix. 2. 2 T HEUAT F ORMAT OF TRANSFORMER-BASED LLM S Multi-Head Attention ( )+ MLP( ) MLP( ) +FFNTransformer （ TF ） Figure 1: The process of Transformer. Current LLMs are primarily based on the memory-enhanced Transformer architecture. In UAT2LLMs Wang & Li (2024b), it has already been demonstrated that the mathematical structure 2