Deep Learning, Artificial Intelligence, NLP, Language Model, Nce.
approaches the likelihood gradient of the normalized model. The model is optimized by maximizing the log-posterior probability of the correct labels D averaged over the data and noise samples: where the expectation over the noise distribution is approximated by sampling. And the gradient is: where