Poincaré Embeddings for Learning Hierarchical Representations
Representation learning has become an invaluable approach for learning from symbolic data such as text and graphs. However, state-of-the-art embedding methods typically do not account for latent hierarchical structures which are characteristic for many complex symbolic datasets. In this work, we introduce a new approach for learning hierarchical representations of symbolic data by embedding them into hyperbolic space -or more precisely into an n-dimensional Poincaré ball. Due to the underlying hyperbolic geometry, this allows us to learn parsimonious representations of symbolic data by simultaneously capturing hierarchy and similarity. We present an efficient algorithm to learn the embeddings based on Riemannian optimization and show experimentally that Poincaré embeddings can outperform Euclidean embeddings significantly on data with latent hierarchies, both in terms of representation capacity and in terms of generalization ability.
Summary
=======
  The paper proposes a link prediction model that embeds symbols in a
  hyperbolic space using Poincaré embeddings. In this space, tree
  structures can more easily be represented as the distance to points
  increases exponentially w.r.t. Euclidean distance. The paper is
  motivated and written well. Furthermore, the presented method is
  intriguing and I believe it will have a notable impact on link
  prediction research. My concerns are regarding the comparison to
  state-of-the-art link prediction and how the method performs if the
  assumption about a hierarchy in the data is dropped.
Strengths
=========
  - Impressive results for embedding hierarchies in a non-Euclidean
    space using a much smaller dimension.
Weaknesses
==========
  - Maybe I am missing a point, but why is there no comparison with
    state-of-the-art neural link prediction models (e.g. HolE/ComplEx)?
    This would strengthen the experimental evaluation a lot as TransE
    has consistently be outperformed by these methods. Furthermore, how
    does Poincaré compare to TransE in terms of runtime?
  - I would have liked to see more experimental results of the
    limitations of the proposed method. Given that many real-world
    datasets have less pronounced hierarchies than WordNet, it would
    have been nice to show what happens if a hierarchy of symbols cannot
    be assumed for a large portion of the data. For instance, how would
    the method perform on FB15k? Generally, I would like to see a more
    thorough discussion about when Poincaré embeddings can be used in
    practice and when not. For instance, the authors mention word2vec,
    GloVe and FastText in the introduction and I guess that Poincaré
    embeddings cannot serve as drop-in replacement for word vectors in
    these models as we don't know how to perform certain translations in
    hyperbolic space or how to apply backprop? Such limitations should
    be clarified in the paper.
Minor Comments
==============
  - Embedding hierarchies in Euclidean has been attempted in prior work
    that should be mentioned:
    - For textual entailment and images: Ivan Vendrov, Ryan Kiros, Sanja
      Fidler, Raquel Urtasun. Order-Embeddings of Images and
      Language. ICLR 2016
    - For link prediction: Thomas Demeester, Tim Rocktäschel, Sebastian
      Riedel. Lifted Rule Injection for Relation Embeddings. EMNLP 2016
  - Footnote 2: "would be more more difficult" -> "would be more
    difficult"
  - L277: "representations whats allow us to" -> "representations that
    allow us to"