Online dictionary extension of word vectors techniques and systems are described that are configured to provide online extension of existing word vector dictionaries and thus overcome the failures of conventional techniques. In one example, a dictionary extension system is employed by a computing system to extend a word vector dictionary to incorporate a new word in an online manner. Co-occurrence information is estimated for the new word with respect to the words in the existing dictionary. This is done by estimating co occurrence information with respect to a large word set based on the existing dictionary and sparse co-occurrence information for the new word. The estimated co-occurrence information is utilized to estimate a new word vector associated with the new word by projecting the estimated co-occurrence information onto the existing word vector dictionary. An extended dictionary is created incorporating the original dictionary and the estimated new word vector. Computing System 102 Word Vector Training System 104 Word Vectors 106 G)) New Wod _1 Dictionary Extension tworSystem 110 _8 Vector Approximation Module 116 Extended Dictionary 118 New Word Vector 1_1_4 14) Word Vectors 106 aurlLanguage Processing System
INCREMENTAL LEARNING OF POINTWISE MUTUAL INFORMATION (PMI) WORD-VECTOR EMBEDDING FOR TEXT/LANGUAGE MODELING