A Coding-Theoretic Analysis of Hyperspherical Prototypical Learning Geometry
<|reference_start|>A Coding-Theoretic Analysis of Hyperspherical Prototypical Learning Geometry: Hyperspherical Prototypical Learning (HPL) is a supervised approach to representation learning that designs class prototypes on the unit hypersphere. The prototypes bias the representations to class separation in a scale invariant and known geometry. Previous approaches to HPL have either of the following shortcomings: (i) they follow an unprincipled optimisation procedure; or (ii) they are theoretically sound, but are constrained to only one possible latent dimension. In this paper, we address both shortcomings. To address (i), we present a principled optimisation procedure whose solution we show is optimal. To address (ii), we construct well-separated prototypes in a wide range of dimensions using linear block codes. Additionally, we give a full characterisation of the optimal prototype placement in terms of achievable and converse bounds, showing that our proposed methods are near-optimal.<|reference_end|>
@article{lindström2024a,
    title={A Coding-Theoretic Analysis of Hyperspherical Prototypical Learning
  Geometry},
    author={Martin Lindstr"om, Borja Rodr'iguez-G'alvez, Ragnar Thobaben,
  Mikael Skoglund},
    journal={arXiv preprint arXiv:2407.07664},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.07664},
    primaryClass={cs.LG cs.AI cs.CV eess.SP stat.ML}
}