A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks
<|reference_start|>A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks: To investigate neural network parameters, it is easier to study the distribution of parameters than to study the parameters in each neuron. The ridgelet transform is a pseudo-inverse operator that maps a given function $f$ to the parameter distribution $\gamma$ so that a network $\mathtt{NN}[\gamma]$ reproduces $f$, i.e. $\mathtt{NN}[\gamma]=f$. For depth-2 fully-connected networks on a Euclidean space, the ridgelet transform has been discovered up to the closed-form expression, thus we could describe how the parameters are distributed. However, for a variety of modern neural network architectures, the closed-form expression has not been known. In this paper, we explain a systematic method using Fourier expressions to derive ridgelet transforms for a variety of modern networks such as networks on finite fields $\mathbb{F}_p$, group convolutional networks on abstract Hilbert space $\mathcal{H}$, fully-connected networks on noncompact symmetric spaces $G/K$, and pooling layers, or the $d$-plane ridgelet transform.<|reference_end|>
@article{sonoda2024a,
    title={A unified Fourier slice method to derive ridgelet transform for a
  variety of depth-2 neural networks},
    author={Sho Sonoda, Isao Ishikawa, Masahiro Ikeda},
    journal={Journal of Statistical Planning and Inference, 2024},
    year={2024},
    doi={10.1016/j.jspi.2024.106184},
    archivePrefix={arXiv},
    eprint={2402.15984},
    primaryClass={cs.LG math.FA stat.ML}
}