Geometric Analysis of Unconstrained Feature Models with $d=K$
<|reference_start|>Geometric Analysis of Unconstrained Feature Models with $d=K$: Recently, interesting empirical phenomena known as Neural Collapse have been observed during the final phase of training deep neural networks for classification tasks. We examine this issue when the feature dimension d is equal to the number of classes K. We demonstrate that two popular unconstrained feature models are strict saddle functions, with every critical point being either a global minimum or a strict saddle point that can be exited using negative curvatures. The primary findings conclusively confirm the conjecture on the unconstrained feature models in previous articles.<|reference_end|>
@article{shen2024geometric,
    title={Geometric Analysis of Unconstrained Feature Models with $d=K$},
    author={Yi Shen and Shao Gu},
    journal={arXiv preprint arXiv:2407.10702},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.10702},
    primaryClass={cs.LG}
}