Deep Learning, Artificial Intelligence, NLP, Language Model, Nce.
the gradient involves a sum over k noise samples instead of a sum over the entire vocabulary, making the NCE training time linear in the number of noise samples and independent of the vocabulary size. 2.2. Log-Bilinear Language (LBL) Models (The models here are not the main point to learn NCE.) In