This work presents the Poincaré Wasserstein Autoencoder, a reformulation of the recently proposed Wasserstein autoencoder framework on a non-Euclidean manifold, the Poincaré ball model of the hyperbolic space H n . By assuming the latent space to be hyperbolic, we can use its intrinsic hierarchy to impose structure on the learned latent space representations. We show that for datasets with latent hierarchies, we can recover the structure in a low-dimensional latent space. We also demonstrate the model in the visual domain to analyze some of its properties and show competitive results on a graph link prediction task.
*Paper summary*
The authors endow a variational auto-encoder with a hyperbolic space for the latent space. This enables them to model hierarchically ranked relationships in the data in a natural way. They use a Wasserstein formulation instead of the standard ELBO, which they claim reduces the variance of the gradients during training.
*Paper decision*
Thanks for an interesting paper, I enjoyed reading it. While there are parts of this paper I like very much and I think it is technically sound, I feel the experiments section is lacking somewhat (see comments below). I therefore am recommending a weak reject; although, upon a favourable rebuttal would happily upgrade this position.
*Supporting arguments*
- In the introduction is well written and the motivation for the introduction of a hyperbolic later space is well laid out.
- The section on Hyperbolic geometry is very well written indeed and presents this non-trivial material in a very simple to understand manner. Maybe what it lacks a bit is motivation for a machine learning audience, for why these mathematical tools from differential geometry would be useful or needed by the community to tackle presenting machine learning problems.
- The structure of the method is well laid out and every hyperbolic version of an auto-encoder operation is explained. I would prefer to see methods from cited works, which are deployed in this paper, written in the paper instead of just referred to, so that I don’t have to search through the cited works to find out how a key operation works.
- I think the method itself is technically sound. 
- I am not sure about the novelty of the method, seeing that there is a very similar paper from earlier this year (Mathieu et al. 2019).
- The experiments are lacking a little. I do not feel that they show a clear reason why a hyperbolic latent space should be used and compared to standard methods, they do not perform any. better. It would have been nice to have a more structured discussion on the merits of the method.
*Questions/notes for the authors*
- Section 3.2, please explain the significance of \lambda_x^c
- Please explain in the related work and/or introduce in more detail than you already have the differences and similarities between this work and your nearest neighbour “Hierarchical representations with poincaré variational auto-encoders” by Mathieu et al. (2019). I would like to know the advantages/disadvantages of using a Wasserstein formulation instead of a sampled ELBO.
- In section 3.3, please point readers forward to the Appendix for a list of gyrovector operations.
- Please use equation numbers on all equations. 
- In your section on “Dispersion representation” you state: “Since the maximum mean discrepancy can be estimated via samples, we do not require a closed form definition of the posterior density as is the case with training using the evidence lower bound. This allows the model to learn richer latent space representations.” Why exactly does not using a closed form posterior density lead to a richer latent space representation? Please back up this statement.
- In the presentation of the ELBO, there is a typo after the second equality where p(z) is listed twice.
- This may just be personal preference on my behalf, but I think a short treatment on VAEs and optimal transport would be useful in the related work and background sections.
- Experimentally it would have been nice to have a direct comparison between this method and Mathieu et al. (2019), that paper being very similar to this one. At least a shared experiment would have been useful.