  Neural word representations have proven useful in Natural Language Processing
(NLP) tasks due to their ability to efficiently model complex semantic and
syntactic word relationships. However, most techniques model only one
representation per word, despite the fact that a single word can have multiple
meanings or "senses". Some techniques model words by using multiple vectors
that are clustered based on context. However, recent neural approaches rarely
focus on the application to a consuming NLP algorithm. Furthermore, the
training process of recent word-sense models is expensive relative to
single-sense embedding processes. This paper presents a novel approach which
addresses these concerns by modeling multiple embeddings for each word based on
supervised disambiguation, which provides a fast and accurate way for a
consuming NLP model to select a sense-disambiguated embedding. We demonstrate
that these embeddings can disambiguate both contrastive senses such as nominal
and verbal senses as well as nuanced senses such as sarcasm. We further
evaluate Part-of-Speech disambiguated embeddings on neural dependency parsing,
yielding a greater than 8% average error reduction in unlabeled attachment
scores across 6 languages.

Computation and Language;Machine Learning