Machine Learning, Naturallanguageprocessing, Artificial Intelligence, NLP, AI.
Referring to our previous example of (orange, juice), we don’t try to predict the probability for juice to be a nearby word i.e P(juice|orange), we try to predict whether (orange, juice) are nearby words or not by calculating P(1|<orange, juice>). So instead of having one giant softmax —