Machine Learning, NLP, Deep Learning, Artificial Intelligence, AI.
the choice of the next word by picking the word corresponding to the maximum probability from the output of the softmax function. This “picking” operation is non-differentiable. Why is this an issue? It’s an issue because, in order to train the generator to minimize 1 - D(G(z)), we need to feed the