Hello Ruggero!



 Ruggero_Ragonesi:

I am trying to backpropagate through a Bernoulli sample.
…
However when I check gradients wrt to parameter, i.e. parameter.grad, they are None.


Sampling from a discrete distribution isn’t differentiable.  So pytorch’s
autograd won’t (and can’t) backpropagate any gradients from
subsequent functions back through the Bernoulli-sampling step.
The value returned from sampling a Bernoulli distribution (or any
discrete distribution) is a discrete value – 0.0 or 1.0.  So you can’t
differentiate it.  That is, you can’t make your sample value a little
smaller, say 0.99 instead of 1.0, by  making your parameter prob
a little smaller.
Best.
K. Frank