Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machines
<|reference_start|>Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machines: We present a constructive universal approximation theorem for learning machines equipped with joint-group-equivariant feature maps, based on the group representation theory. ``Constructive'' here indicates that the distribution of parameters is given in a closed-form expression known as the ridgelet transform. Joint-group-equivariance encompasses a broad class of feature maps that generalize classical group-equivariance. Notably, this class includes fully-connected networks, which are not group-equivariant but are joint-group-equivariant. Moreover, our main theorem also unifies the universal approximation theorems for both shallow and deep networks. While the universality of shallow networks has been investigated in a unified manner by the ridgelet transform, the universality of deep networks has been investigated in a case-by-case manner.<|reference_end|>
@article{sonoda2024unified,
    title={Unified Universality Theorem for Deep and Shallow
  Joint-Group-Equivariant Machines},
    author={Sho Sonoda, Yuka Hashimoto, Isao Ishikawa, Masahiro Ikeda},
    journal={arXiv preprint arXiv:2405.13682},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.13682},
    primaryClass={cs.LG math.RT stat.ML}
}