Deep Learning, Artificial Intelligence, Neural Networks, Word Prediction, Language Model.
perplexity of 196.6, a 3.8% improvement. Total improvement in test perplexity with respect to the trigram baseline is 29%. The required number of samples with the non-adaptive unigram was growing exponentially. Reference [2007 TNN] [Bengio TNN’07] Adaptive Importance Sampling to Accelerate Training