Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input
<|reference_start|>Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input: In this work, we study the mean-field flow for learning subspace-sparse polynomials using stochastic gradient descent and two-layer neural networks, where the input distribution is standard Gaussian and the output only depends on the projection of the input onto a low-dimensional subspace. We propose a basis-free generalization of the merged-staircase property in Abbe et al. (2022) and establish a necessary condition for the SGD-learnability. In addition, we prove that the condition is almost sufficient, in the sense that a condition slightly stronger than the necessary condition can guarantee the exponential decay of the loss functional to zero.<|reference_end|>
@article{chen2024mean-field,
    title={Mean-Field Analysis for Learning Subspace-Sparse Polynomials with
  Gaussian Input},
    author={Ziang Chen, Rong Ge},
    journal={arXiv preprint arXiv:2402.08948},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.08948},
    primaryClass={cs.LG math.AP}
}