Solved by moher in post #2 
              
              
                I fixed it myself. 
when updating the weights, I have to put them in with torch.no_grad(): 
as apparently not doing so makes the computational graph acyclic. (am I write?) 
Also, it is correct to use w.grad.zero_() not w.grad.data.zero(), which I can’t understand why yet! 
So, the corrected part is …