Machine Learning, Optimization, Gradient Descent, Reinforcement Learning.
but backprop using the entire (dense) vector. In general, this approximation is called “straight through.” On paper it might look a bit sketchy but empirically it works pretty well compared to REINFORCE methods. The above diagram shows the main crux of Gumbel softmax. The action probabilities π are