  Hyperbolic neural networks have shown great potential for modeling complex
data. However, existing hyperbolic networks are not completely hyperbolic, as
they encode features in a hyperbolic space yet formalize most of their
operations in the tangent space (a Euclidean subspace) at the origin of the
hyperbolic space. This hybrid method greatly limits the modeling ability of
networks. In this paper, we propose a fully hyperbolic framework to build
hyperbolic networks based on the Lorentz model by adapting the Lorentz
transformations (including boost and rotation) to formalize essential
operations of neural networks. Moreover, we also prove that linear
transformation in tangent spaces used by existing hyperbolic networks is a
relaxation of the Lorentz rotation and does not include the boost, implicitly
limiting the capabilities of existing hyperbolic networks. The experimental
results on four NLP tasks show that our method has better performance for
building both shallow and deep networks. Our code will be released to
facilitate follow-up research.

Computation and Language;Machine Learning