Activation thresholds and expressiveness of polynomial neural networks
<|reference_start|>Activation thresholds and expressiveness of polynomial neural networks: Polynomial neural networks have been implemented in a range of applications and present an advantageous framework for theoretical machine learning. A polynomial neural network of fixed architecture and activation degree gives an algebraic map from the network's weights to a set of polynomials. The image of this map is the space of functions representable by the network. Its Zariski closure is an affine variety known as a neurovariety. The dimension of a polynomial neural network's neurovariety provides a measure of its expressivity. In this work, we introduce the notion of the activation threshold of a network architecture which expresses when the dimension of a neurovariety achieves its theoretical maximum. In addition, we prove expressiveness results for polynomial neural networks with equi-width~architectures.<|reference_end|>
@article{finkel2024activation,
    title={Activation thresholds and expressiveness of polynomial neural networks},
    author={Bella Finkel, Jose Israel Rodriguez, Chenxi Wu, Thomas Yahl},
    journal={arXiv preprint arXiv:2408.04569},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.04569},
    primaryClass={cs.LG cs.NE math.AG stat.ML}
}