Ops, sorry got the code moved, but yes, essentially that is the correct notebook. I am trying to implement your recommendations, and I think this confused me even more.
What I am trying to do, is to calculate the gradient of loss w.r.t w1 and update it. My y values are the expected Q_target and my y_pred is just Q from the prior step.
Looking at the other tutorials, they don’t provide anything in backward, but the backward() works on all Variables with requires_grad=True.
Based on that, I thought as long as w1 present in the computations its gradients will be computed. Is that not the case?
Edit: also, as I am trying multiple iterations on the code here 3 it seems to be producing a good result, but why?