Machine Learning, Optimization, Gradient Descent, Reinforcement Learning.
for 2 reasons. First, the Gumbel softmax makes a sample with k nonzero elements instead of 1 nonzero element. Remember k is the number of possible actions (dimension of action space). Therefore a sample from Gumbel softmax is never exactly a sample from the multinoulli. Second, the expected value