Hi Truss!



 truss712:


Is this approach feasible for PyTorch, where I have to break the graph on purpose?



Probably not.  To make pytorch’s autograd and gradient-descent-based
optimizers work, you would have to “unbreak” the graph, patching it
back together somehow.



Is there a better way to address this issue?



The core problem is that it is hard to make the result of a Monte Carlo
simulation differentiable with respect to your model parameters.
Consider the derivative of the area of a circle with respect to its radius.
Now consider calculating the area with a Monte Carlo “simulation”
(integration).  Sample two-dimensional points uniformly from the square
x in [-1.0, 1.0]; y in [-1.0, 1.0] and count the fraction of
those points that fall in a circle of radius r <= 1.0.  You then have
area = 4 * fraction.  In this scheme, how would you calculate the
derivative of area with respect to r?
You can imagine generating “simple” random numbers (say uniform
or normal), using them to drive a Monte-Carlo process that is
implemented with pytorch differentiable tensor functions, and then
using autograd to compute the gradient of some final loss function
with respect to parameters that characterize the Monte-Carlo process
(as well as with respect to the random numbers, if you so choose).
But whether you can do this in practice depends on the details of
your Monte-Carlo process.



Is it possible if I define my simulation script as a custom torch layer?



Yes, in principle.  (But it may be impractical or impossible for your use
case.)
You would do this by defining a custom autograd function 1 (your “custom
torch layer”).
It will have a forward() method that can do most anything – numpy
calculations, a Monte Carlo simulation, any sort of black box.  But it
also has to have a matching backward() method that calculates the
gradient (or an approximation to, or surrogate for the gradient) of the
output of forward() with respect to its input.
It’s this backward() method that “patches the graph back together.”
Best.
K. Frank