Hello Capo!



 Capo_Mestre:

This probably breaks the flow of computation graph,


You are correct; this does break the computation graph.


but I am not sure how to make minimize work otherwise.


In order to be able to backpropagate through the minimize() part
of your loss-function computation, pytorch has to be able to get the
gradient of minimize()'s output with respect to its input.
You have two choices:
You can write the minimize()* logic entirely with pytorch tensor
operations, and let pytorch’s autograd calculate the gradients for
you.
Or you can wrap the computation that uses minimize() in a
torch.autograd.Function, work out the gradient of minimize()
“by hand,” and implement that gradient computation in the
.backward() method of your torch.autograd.Function.
(Both approaches are likely to entail significant work on your part.
If I were doing it, I would be lazy and try to cook up a loss function
that doesn’t use minimize(), but still captures the important part
of what minimize() is doing for you in some approximate way.)
*)  With the caveat that minimize() is a relatively complicated
iterative algorithm, which can require care writing so that it builds a
proper computation graph, and is likely to generate large, expensive
computation graphs.
Good luck.
K. Frank