A Margin-based Multiclass Generalization Bound via Geometric Complexity
<|reference_start|>A Margin-based Multiclass Generalization Bound via Geometric Complexity: There has been considerable effort to better understand the generalization capabilities of deep neural networks both as a means to unlock a theoretical understanding of their success as well as providing directions for further improvements. In this paper, we investigate margin-based multiclass generalization bounds for neural networks which rely on a recent complexity measure, the geometric complexity, developed for neural networks. We derive a new upper bound on the generalization error which scales with the margin-normalized geometric complexity of the network and which holds for a broad family of data distributions and model classes. Our generalization bound is empirically investigated for a ResNet-18 model trained with SGD on the CIFAR-10 and CIFAR-100 datasets with both original and random labels.<|reference_end|>
@article{munn2024a,
    title={A Margin-based Multiclass Generalization Bound via Geometric Complexity},
    author={Michael Munn, Benoit Dherin, Javier Gonzalvo},
    journal={Proceedings of 2nd Annual Workshop on Topology, Algebra, and
  Geometry in Machine Learning (TAG-ML), PMLR 221:189-205, 2023},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.18590},
    primaryClass={stat.ML cs.LG}
}