Analytical Solution of a Three-layer Network with a Matrix Exponential Activation Function
<|reference_start|>Analytical Solution of a Three-layer Network with a Matrix Exponential Activation Function: In practice, deeper networks tend to be more powerful than shallow ones, but this has not been understood theoretically. In this paper, we find the analytical solution of a three-layer network with a matrix exponential activation function, i.e., $$ f(X)=W_3\exp(W_2\exp(W_1X)), X\in \mathbb{C}^{d\times d} $$ have analytical solutions for the equations $$   Y_1=f(X_1),Y_2=f(X_2) $$ for $X_1,X_2,Y_1,Y_2$ with only invertible assumptions. Our proof shows the power of depth and the use of a non-linear activation function, since one layer network can only solve one equation,i.e.,$Y=WX$.<|reference_end|>
@article{gai2024analytical,
    title={Analytical Solution of a Three-layer Network with a Matrix Exponential
  Activation Function},
    author={Kuo Gai, Shihua Zhang},
    journal={arXiv preprint arXiv:2407.02540},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.02540},
    primaryClass={stat.ML cs.AI cs.LG}
}