Solved by tom in post #2 
              
              
                You need to declare your parameters in the __init__, but put all computation (except for constants) in the forward pass. 
You shouldn’t have to do requires_grad_ in the forward. 
You should not use tensor or Tensor within your calculation. This will break the graph. 
It isn’t entirely clear to me wh…