Machine Learning, Naturallanguageprocessing, Artificial Intelligence, NLP, AI.
all the weights in the hidden layer. The weight corresponding to non-target words would receive a marginal or no change at all, i.e. in each pass we only make very sparse updates. Secondly, for every training sample, the calculation of the final probabilities using the softmax is quite an expensive