This paper introduces an approach to learn representations based on the Lorentzian distance in hyperbolic geometry. Hyperbolic geometry is especially suited to hierarchically-structured datasets, which are prevalent in the real world. Current hyperbolic representation learning methods compare examples with the Poincaré distance metric. They formulate the problem as minimizing the distance of each node in a hierarchy with its descendants while maximizing its distance with other nodes. This formulation produces node representations close to the centroid of their descendants. We exploit the fact that the centroid w.r.t the squared Lorentzian distance can be written in closed-form. We show that the Euclidean norm of such a centroid decreases as the curvature of the hyperbolic space decreases. This property makes it appropriate to represent hierarchies where parent nodes minimize the distances to their descendants and have smaller Euclidean norm than their children. Our approach obtains state-of-the-art results in retrieval and classification tasks on different datasets.
Summary
Learning embeddings of graphs in hyperbolic space have become popular and yielded promising results. A core reason for that is learning hierarchical representations of the graphs is easier in hyperbolic space due to the curvature and the geometrical properties of the hyperbolic space. Similar to [1, 2], this paper uses Lorentzian model of the hyperbolic space in order to learn embeddings of the graph. The main difference of the proposed approach in this paper is that  they come up with a closed-form solution such that each node representation close to the centroid of their descendants. A curious property of the equation for the centroids proposed to learn the embeddings of each node also encodes information related to the specificity in the Euclidean norm of the centroid. Also this paper introduces two additional hyperparameters. Beta hyperparameter is selected to control the curvature of the space. Depending on the task the optimal curvature can be tuned to be a different value. This also ties closely with the de-Sitter spaces. Authors provide results on different graph embedding benchmark tasks. The paper claims that, an advantage of the proposed approach is that the embedding of the model can be tuned with regular SGD without needing to use Riemannian optimization techniques.
Questions
Have you tried learning beta instead of selecting as a hyperparameter?
The paper claims that Riemannian optimization is not necessary for this model, but have you tried optimizing the model with the Riemannian optimization methods?
Equation 11, bears lots of similarity to the Einstein gyro-midpoint method proposed by Abraham Ungar which is also used by [2]. Have you investigated the relationship between the two formulations?
On Eurovoc dataset the results of the proposed method is worse than the d_P in H^d. Do you have a justification of why that happens?
Pros
The paper delivers some interesting theoretical findings about the embeddings learned in hyperbolic space, e.g. a closed for equation in the
The paper is written well. The goal and motivations are clear.
Cons
Experiments are only limited to small scale-traditional graph datasets. It would be more interesting to see how those embeddings would perform on large-scale datasets such as to learn knowledge-base embeddings or for recommendation systems.
Although the idea is interesting. Learning graph embeddings have already been explored in [1]. The main contribution of this paper is thus mainly focuses on the close-form equation for the centroid and the curvature hyperparameter. These changes provide significant improvements on the results but still the novelty of the approach is in that sense limited compared to [1].
Minor comment:
It is really difficult to understand what is in Figure 2 and 3. Can you reduce the number of data points and just emphasize a few nodes in the graph that shows a clear hierarchy.
[1] Nickel, Maximilian, and Douwe Kiela. "Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry." arXiv preprint arXiv:1806.03417 (2018).
[2] Gulcehre, Caglar, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia et al. "Hyperbolic Attention Networks." arXiv preprint arXiv:1805.09786 (2018).