Fundamental operating regimes, hyper-parameter fine-tuning and glassiness: towards an interpretable replica-theory for trained restricted Boltzmann machines
<|reference_start|>Fundamental operating regimes, hyper-parameter fine-tuning and glassiness: towards an interpretable replica-theory for trained restricted Boltzmann machines: We consider restricted Boltzmann machines with a binary visible layer and a Gaussian hidden layer trained by an unlabelled dataset composed of noisy realizations of a single ground pattern. We develop a statistical mechanics framework to describe the network generative capabilities, by exploiting the replica trick and assuming self-averaging of the underlying order parameters (i.e., replica symmetry). In particular, we outline the effective control parameters (e.g., the relative number of weights to be trained, the regularization parameter), whose tuning can yield qualitatively-different operative regimes. Further, we provide analytical and numerical evidence for the existence of a sub-region in the space of the hyperparameters where replica-symmetry breaking occurs.<|reference_end|>
@article{fachechi2024fundamental,
    title={Fundamental operating regimes, hyper-parameter fine-tuning and
  glassiness: towards an interpretable replica-theory for trained restricted
  Boltzmann machines},
    author={Alberto Fachechi, Elena Agliari, Miriam Aquaro, Anthony Coolen, Menno
  Mulder},
    journal={arXiv preprint arXiv:2406.09924},
    year={2024},
    number={Roma01.Math},
    archivePrefix={arXiv},
    eprint={2406.09924},
    primaryClass={cond-mat.dis-nn cond-mat.stat-mech cs.LG}
}