Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension
<|reference_start|>Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension: Bounds on the smallest eigenvalue of the neural tangent kernel (NTK) are a key ingredient in the analysis of neural network optimization and memorization. However, existing results require distributional assumptions on the data and are limited to a high-dimensional setting, where the input dimension $d_0$ scales at least logarithmically in the number of samples $n$. In this work we remove both of these requirements and instead provide bounds in terms of a measure of the collinearity of the data: notably these bounds hold with high probability even when $d_0$ is held constant versus $n$. We prove our results through a novel application of the hemisphere transform.<|reference_end|>
@article{karhadkar2024bounds,
    title={Bounds for the smallest eigenvalue of the NTK for arbitrary spherical
  data of arbitrary dimension},
    author={Kedar Karhadkar, Michael Murray, Guido Mont'ufar},
    journal={arXiv preprint arXiv:2405.14630},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.14630},
    primaryClass={stat.ML cs.LG}
}