If you consider loss minimization, you can consider the (negative) gradient as a hint “go this way with your parameters a tiny bit to reduce the loss”. Now if you can apply the reparametrisation trick, you can say that tweaking the parameters a little bit also moves the output by small amount. This relationship enables you to backpropagate.
However, if you have, say a bernoulli random variable (i.e. 1 with probability p, 0 with probablity (1-p)), you cannot “wiggle” with the outcomes - they’ll be 0 or 1.
The basic trick there is to recognize that you are interested in the expectation rather than individual sample and then to figure out how to exchange expectation and derivative for backpropagation and make an expression with an expectation from the result - this is a key to the REINFORCE-type methods.
Best regards
Thomas