IMPOSING CATEGORY TREES ONTO WORD- EMBEDDINGS USING A GEOMETRIC CONSTRUCTION
We present a novel method to precisely impose tree-structured category information onto word-embeddings, resulting in ball embeddings in higher dimensional spaces (N -balls for short). Inclusion relations among N -balls implicitly encode subordinate relations among categories. The similarity measurement in terms of the cosine function is enriched by category information. Using a geometric construction method instead of back-propagation, we create large N -ball embeddings that satisfy two conditions: (1) category trees are precisely imposed onto word embeddings at zero energy cost; (2) pre-trained word embeddings are well preserved. A new benchmark data set is created for validating the category of unknown words. Experiments show that N -ball embeddings, carrying category information, significantly outperform word embeddings in the test of nearest neighborhoods, and demonstrate surprisingly good performance in validating categories of unknown words. Source codes and data-sets are free for public access https://github.com/GnodIsNait/nball4tree.git and https://github.com/GnodIsNait/bp94nball.git.
Attention!!! This submission contains Github and Google Drive links to author-related accounts (see e.g. the abstract). I do not think this is permitted or standard. I leave the decision regarding "automatic rejection" of the submission to meta-reviewers of the paper.
------------------------------------------------
The paper presents a method for tweaking existing vector embeddings of categorical objects (such as words), to convert them to ball embeddings that follow hierarchies. Each category is represented as a Eucldiean norm ball in high dimensional space, with center and radii adaptable to data. Next, inclusion and exclusion constraints on each pair of  balls are imposed based on the hierarchical structure. These constraints are imposed via an algorithmic approach.  The empirical study includes investigating the consistency of the representation with the hierarchy and demonstrating nearest neighbors for a set of words.
On the positive side, the paper addresses an important problem. It is readable and well organized. The related work could be improved by adding a number of representative related works such as [3,4].  
The major concern about the paper is the originality of the method. Encoding hierarchies with high dimensional balls and encoding inclusion and exclusion as constraints on those balls is a neat and powerful idea from modeling perspective. However, it is not novel, since the approach is already established for example in [1, 2 Chapter 5]. 
The next major concern is regarding the evaluation of the quality of embeddings.
The empirical evaluation does not sufficiently evaluate the quality of tweaked embeddings. In contrast, the quantitative evaluation is more concerned with if the embeddings being consistent with the given hierarchy. In particular, not enough quantitative evidence that the proposed embeddings are actually effective in capturing semantics or in prediction tasks is provided. It should be noted that, the first aspect, ie consistency of the feasible solutions with hierarchy, can be theoretically established (see e.g. [1]).  The first paragraph of 3.2 seems unclear or wrong. See for example [2] for a gradient based solution for the problem.
Finally, using an algorithmic approach as opposed to learning method for constructing embeddings, makes the method not directly related to the topic of ICLR conference.
Overall, due to the above reasons, I vote the paper to be rejected. (The poor anonymization makes it a strong case for a reject.)
[1] Mirzazadeh, F., Ravanbakhsh S., Ding N., Schuurmans D.,  "Embedding inference for structured multilabel prediction", NIPS 2015.
[2] Mirzazadeh, F."Solving Association Problems with Convex Co-embedding", PhD thesis, 2017. (Chapter 5)
[3] Vilnis, Luke, and Andrew McCallum. "Word representations via gaussian embedding.", ICLR 2015.
[4] Vendrov, I., Kiros, R., Fidler, S., Urtasun, R. "Order-embeddings of images and language." ICLR 2016.