Machine Learning, Optimization, Gradient Descent, Reinforcement Learning.
Optimizing the baseline to lower the variance of the REINFORCE term runs the risk of increasing the variance of the Reparameterization term. Therefore let’s directly minimize the variance of the gradient, which is exactly the metric we want to reduce. In fact, we’ll see an example later where