Gyrogroup Batch Normalization
Ziheng Chen Yue Song Xiaojun Wu Nicu Sebe
Several Riemannian manifolds in machine learning, such as Symmetric Positive Definite (SPD), Grassmann, spherical, and hyperbolic manifolds, have been proven to admit gyro structures, thus enabling a principled and effective extension of Euclidean Deep Neural Networks (DNNs) to manifolds. Inspired by this, this study introduces a general Riemannian Batch Normalization (RBN) framework on gyrogroups, termed GyroBN. We identify the least requirements to guarantee GyroBN with theoretical control over sample statistics, referred to as \textit{pseudo-reduction} and \textit{gyroisometric gyrations}, which are satisfied by all the existing gyrogroups in machine learning. Besides, our GyroBN incorporates several existing normalization methods, including the one on general Lie groups and different types of RBN on the non-group SPD geometry. Lastly, we instantiate our GyroBN on the Grassmannian and hyperbolic spaces. Experiments on the Grassmannian and hyperbolic networks demonstrate the effectiveness of our GyroBN. The code is available at https://github.com/GitZH-Chen/GyroBN.git.
Gyrovector Spaces, Riemannian Manifolds, Riemannian Batch Normalization, Grassmannian Manifolds, Hyperbolic Manifolds
This paper proposes a general framework for Riemannian Batch Normalization (RBN) over gyrogroups (GyroBN), which incorporate several existing RBN methods, and showcase our framework on the Grassmannian and hyperbolic geometries.