Deep Manifold Part 1: Anatomy of Neural Network Manifold
<|reference_start|>Deep Manifold Part 1: Anatomy of Neural Network Manifold: Based on the numerical manifold method principle, we developed a mathematical framework of a neural network manifold: Deep Manifold and discovered that neural networks: 1) is numerical computation combining forward and inverse; 2) have near infinite degrees of freedom; 3) exponential learning capacity with depth; 4) have self-progressing boundary conditions; 5) has training hidden bottleneck. We also define two concepts: neural network learning space and deep manifold space and introduce two concepts: neural network intrinsic pathway and fixed point. We raise three fundamental questions: 1). What is the training completion definition; 2). where is the deep learning convergence point (neural network fixed point); 3). How important is token timestamp in training data given negative time is critical in inverse problem.<|reference_end|>
@article{ma2024deep,
    title={Deep Manifold Part 1: Anatomy of Neural Network Manifold},
    author={Max Y. Ma and Gen-Hua Shi},
    journal={arXiv preprint arXiv:2409.17592},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.17592},
    primaryClass={cs.LG cs.AI}
}