LEARNING MIXED-CURVATURE REPRESENTATIONS IN PRODUCTS OF MODEL SPACES
The quality of the representations achieved by embeddings is determined by how well the geometry of the embedding space matches the structure of the data. Euclidean space has been the workhorse for embeddings; recently hyperbolic and spherical spaces have gained popularity due to their ability to better embed new types of structured data-such as hierarchical data-but most data is not structured so uniformly. We address this problem by proposing learning embeddings in a product manifold combining multiple copies of these model spaces (spherical, hyperbolic, Euclidean), providing a space of heterogeneous curvature suitable for a wide variety of structures. We introduce a heuristic to estimate the sectional curvature of graph data and directly determine an appropriate signature-the number of component spaces and their dimensions-of the product manifold. Empirically, we jointly learn the curvature and the embedding in the product space via Riemannian optimization. We discuss how to define and compute intrinsic quantities such as means-a challenging notion for product manifolds-and provably learnable optimization functions. On a range of datasets and reconstruction tasks, our product space embeddings outperform single Euclidean or hyperbolic spaces used in previous works, reducing distortion by 32.55% on a Facebook social network dataset. We learn word embeddings and find that a product of hyperbolic spaces in 50 dimensions consistently improves on baseline Euclidean and hyperbolic embeddings, by 2.6 points in Spearman rank correlation on similarity tasks and 3.4 points on analogy accuracy.
Page 2: What are p_i, i=1,2,...,n, their set T and \mathcal{P}?
What is | | used to compute distortion between a and b?
Please fix the definition of the Riemannian manifold, such that M is not just any manifold, but should be a smooth manifold or a particular differentiable manifold. Please update your definition more precisely, by checking page 328 in J.M. Lee, Introduction to Smooth Manifolds, 2012, or Page 38 in do Cormo, Riemannian Geometry, 1992.
Please define \mathcal{P} in equation (1).
Define K used in the definition of the hyperboloid more precisely.
Please provide proofs of these statements for product of manifolds with nonnegative and nonpositive curvatures: “In particular, the squared distance in the product decomposes via (1). In other words, dP is simply the l2 norm of the component distances dMi.”
Please explain what you mean by “without the need for optimization” in “These distances provide simple and interpretable embedding spaces using P, enabling us to introduce combinatorial constructions that allow for embeddings without the need for optimization.” In addition, how can you compute geodesic etc. if you use l1 distance for the embedded space?
By equation (2), the paper focuses on embedding graphs, which is indeed the main goal of the paper. Therefore, first, the novelty and claims of the paper should be revised for graph embedding. Second, three particular spaces are considered in this work, which are the sphere, hyperbolic manifold, and Euclidean space. Therefore, you cannot simply state your novelty for a general class of product spaces. Thus, the title, novelty, claims and other parts of the paper should be revised and updated according to the particular input and output spaces of embeddings considered in the paper. 
Please explain how you compute the metric tensor  g_P and apply the Riemannian correction (multiply by the inverse of the metric tensor g_P) to determine the Riemannian gradient in the Algorithm 1, more precisely. 
Step (9) of the Algorithm 1 is either wrong, or you compute v_i without projecting the Riemannian gradient. Please check your theoretical/experimental results and code according to this step.
What is h_i used in the Algorithm 1? Can we suppose that it is the ith component of h?
In step (6) and step (8), do you project individual components of the Riemannian gradient to the product manifold? Since their dimensions are different, how do you perform these projections, since definitions of the projections given on Page 5 cannot be applied? Please check your theoretical/experimental results and code accordingly.
Please define exp_{x^(t)_i}(vi) and Exp(U) more precisely. I suppose that they denote exponential maps.
How do you initialize x^(0) randomly?
The notation is pretty confusing and ambiguous. First, does x belong to an embedded Riemannian manifold P or a point on the graph, which will be embedded? According to equation (2), they are on the graph and they will be embedded. According to Algorithm 1, x^0 belongs to P, which is a Riemannian manifold as defined before. So, if x^(0) belongs to P, then L is already defined from P to R (in input of the Algorithm 1). Thereby, gradient \nabla L(x) is already a Riemannian gradient, not the Euclidean gradient, while you claim that \nabla L(x) is the Euclidean gradient in the text.
Overall, Algorithm 1 just performs a projection of Riemannian or Euclidean gradient  \nabla L(x) onto a point v_i for each ith individual manifold. Then, each v_i is projected back to a point on an individual component of the product manifold by an exponential map. 
What do you mean by “sectional curvature, which is a function of a point p and two directions x; y from p”? Are x and y not points on a manifold?
 
You define \xi_G(m;b,c) for curvature estimation for a graph G. However, the goal was to map G to a Riemannian manifold. Then, do you also consider that G is itself a Riemannian manifold, or a submanifold?
What is P in the statement “the components of
the points in P” in Lemma 2?
What is \epsilon in Lemma 2?
How do you optimize positive w_i, i=1,2,...,n?
What is the “gradient descent” refered to in Lemma 2?
Please provide computational complexity and running time of the methods.
Please define \mathbb{I}_r.
At the third line of the first equation of the proof of Lemma 1, there is no x_2. Is this equation correct?
If at least of two of x1, y1, x2 and y2 are linearly dependents, then how does the result of Lemma 1 change?
Statements and results given in Lemma 1 are confusing. According to the result, e.g. for K=1, curvature of product manifold of sphere S and Euclidean space E is 1, and that of E and hyperbolic  H is 0. Then, could you please explain this result for the product of S, E and H, that is, explain the statement “The last case (one negative, one positive space) follows along the same lines.”? If the curvature of the product manifold is non-negative, then does it mean that the curvature of H is ignored in the computations?
What is \gamma more precisely? Is it a distribution or density function? If it is, then what does (\gamma+1)/2 denote?
The statements related to use of Algorithm 1 and SGD to optimize equation (2) are confusing. Please explain how you employed them together in detail.
Could you please clarify estimation of K_1 and K_2, if they are unknown. More precisely, the following statements are not clear;
- “Furthermore, without knowing K1, K2 a priori, an estimate for these curvatures can be found by matching the distribution of sectional curvature from Algorithm 2 to the empirical curvature computed from Algorithm 3. In particular, Algorithm 2 can be used to generate distributions, and K1, K2 can then be found by matching moments.” Please explain how in more detail? What is matching moments?
- “we find the distribution via sampling (Algorithm 3) in the calculations for Table 3, before being fed into Algorithm 2 to estimate Ki” How do you estimation K_1 and K_2 using Algorithm 3?
- Please define, “random (V)”, “random neighbor m” and “\delta_K/s” used in Algorithm 3 more precisely.