  Continuous symmetries and their breaking play a prominent role in
contemporary physics. Effective low-energy field theories around symmetry
breaking states explain diverse phenomena such as superconductivity, magnetism,
and the mass of nucleons. We show that such field theories can also be a useful
tool in machine learning, in particular for loss functions with continuous
symmetries that are spontaneously broken by random initializations. In this
paper, we illuminate our earlier published work (Bamler & Mandt, 2018) on this
topic more from the perspective of theoretical physics. We show that the
analogies between superconductivity and symmetry breaking in temporal
representation learning are rather deep, allowing us to formulate a gauge
theory of `charged' embedding vectors in time series models. We show that
making the loss function gauge invariant speeds up convergence in such models.

Machine Learning;Statistical Mechanics;Machine Learning