  Distributed word representations have been demonstrated to be effective in
capturing semantic and syntactic regularities. Unsupervised representation
learning from large unlabeled corpora can learn similar representations for
those words that present similar co-occurrence statistics. Besides local
occurrence statistics, global topical information is also important knowledge
that may help discriminate a word from another. In this paper, we incorporate
category information of documents in the learning of word representations and
to learn the proposed models in a document-wise manner. Our models outperform
several state-of-the-art models in word analogy and word similarity tasks.
Moreover, we evaluate the learned word vectors on sentiment analysis and text
classification tasks, which shows the superiority of our learned word vectors.
We also learn high-quality category embeddings that reflect topical meanings.

Computation and Language