Deep Learning, Artificial Intelligence, NLP, Language Model, Nce.
Medium) Noise-Contrastive Estimation (NCE) is another basic concept of Contrastive Learning in self-supervised learning. Outline Softmax in Word2Vec Noise-Contrastive Estimation (NCE) Experimental Results 1. Softmax in Word2Vec In Word2Vec, Softmax is used at the end for CBOW and Skip-gram models: