It’s the basic method behind policy gradient reinforcement learning. It’s referenced in the docs here:
https://pytorch.org/docs/stable/distributions.html 55
This blogpost provides a more elaborate explanation:
http://karpathy.github.io/2016/05/31/rl/ 61
It’s normally used in explicit reinforcement learning settings, when your network produces a distribution over possible actions and the environment provides you with a reward for the action. This is also how it is referenced in the docs. However, it can be used for any model that uses some non-differentiable sampling step. The principle is generalized in this paper 14.
My example is a little artificial: the model produces a distribution on the outputs and then samples from that distribution, computes a loss on the sample, and then estimates a gradient on that sample using REINFORCE.