Many applications of linguistic embedding models rely on their value as pretrained inputs for end-to-end tasks such as dialog modeling, machine translation, or question answering. This position paper presents an alternate paradigm: Rather than using learned embeddings as input features, we instead treat them as a common-sense knowledge repository that can be queried via simple mathematical operations within the embedding space. We show how linear offsets can be used to (a) identify an object given its description, (b) discover relations of an object given its label, and (c) map free-form text to a set of action primitives. Our experiments provide a valuable proof of concept that language-informed common sense reasoning, or 'reasoning in the linguistic domain', lies within the grasp of the research community. In order to attain this goal, however, we must reconsider the way neural embedding models are typically trained an evaluated. To that end, we also identify three empirically-motivated evaluation metrics for use in the training of future embedding models.
This is a position paper - it discusses overall about multi-word embedding (sentence) level and how it can be leveraged in multiple applications. Also, it discusses some open opportunities on how the embeddings can be used as a base knowledge source and some challenges in them
Pros:
1. The examples and analysis of embeddings involved in the paper is detailed
2. The authors have run lots of experiments to position their idea correctly.
Cons:
1. I find many of their intuitions well established in the literature - the intuition of geometric structures in the embedding space, the intuition of incidence angle. The ideas from section 4,5,6 seem to be rather obvious extension of what we know
2. Especially, in Section 4,5 the idea of finding indexes, is nothing but the projection of x on the mean of (A,B) - which is more of a common practice. I find both these ideas of finding indexes and extrapolation, pretty much recurrent and existing in literature.
3. What I would be really interested is, a new training algorithm for learning embeddings, that would be able to answer all linear algebra based questions in the embedding space. Rather, this paper analyses the pros and cons of 6 existing models. In my personal opinion, this is half work done for a very good paper.
4. Personally, I am not very appreciative of the use of  "Common Sense" in this paper. There is a set of papers in the literature (atleast in vision domain), where they actually try to learn the common sense from the underlying data. The use of common sense in this paper does not align with that usage and hence, is misleading for me.
5. From reading a position paper, I would ideally want to see ideas worth of atleast 2-3 PhD thesis. I do not find such compelling ideas in this paper.