Deep Learning, Artificial Intelligence, NLP, Language Model, Nce.
basic idea is to train a logistic regression classifier to discriminate between samples from the data distribution and samples from some “noise” distribution, based on the ratio of probabilities of the sample under the model and the noise distribution. We would like to learn the distribution of