Machine Learning, Optimization, Gradient Descent, Reinforcement Learning.
estimator. You reduce the variance when you subtract the REINFORCE baseline but it’s not clear that this will offset the additional variance from the surrogate Reparameterization term you’ve added. It’s encouraging to check that as c_ϕ approaches f, we get back the Reparameterization estimator