Hi Yigit!



 yildirimyigit:

I did a simple forward pass. But upon receiving the rewards, I had to use detach() to get these values to calculate the loss and now, I cannot backpropagate since I have broken the computation graph when I use detach().
Now, I only see two possibilities: 1) move everything to pytorch and abandon numpy


If you move your loss function from pytorch to numpy (and use
differentiable pytorch tensor functions to calculate your loss),
pytorch’s autograd will calculate the gradients for you when you
run loss.backward().  (You don’t have to move everything
to pytorch, but you do have to move the entire loss function to
pytorch.)



try to come up with a hybrid solution.



If by “hybrid” you mean that you want to keep your loss function
in numpy, then you will have to write an explicit backward()
function (that can be in numpy, if you choose) for your loss
function that calculates its gradients.  Once you use something
other than pytorch tensor functions, you won’t be able to get
the gradients / backward() done automatically for you.
When using a custom loss function, I’ve generally preferred to
try to implement it with tensor functions to get autograd “for free.”
Plus you will get pytorch’s cpu / gpu management, and possibly
useful speed-ups from running both your loss function’s
forward() and backward() on the gpu, if you so choose.
I do believe that you will have to pay one way or another:  Either
port your numpy loss to pytorch; or write backward() explicitly.
Good luck.
K. Frank