Hi Anurag!



 Anurag_Pandey:

I’m just wondering if sampling from other distribution is a differentiable operation or not?


I do not believe that pytorch supports differentiating a sample from a
(continuous) distribution with respect to parameters describing that
distribution.  (But I haven’t checked.)
However, doing so can make mathematical sense, and you can
sometimes structure your calculations so that it works.
Consider a gaussian distribution with mean mu, and standard
deviation sigma.  You can sample from such a distribution by
first sampling a value z from a normal distribution (mean = 0,
standard deviation = 1), and then transforming that value:
x = sigma * z + mu

If mu and sigma are packaged as Tensors that have
requires_grad = True, then calling x.backward() will
compute gradients for sigma and mu.
(Whether you will get the result you expect depends on what you
were expecting, but this scheme is conceptually self-consistent.)
Best.
K. Frank