Damn, thats good to know. My goal is to still update the params in the net, but via ag.grad(), not through loss.backward()… It sounds like from what you are saying that even that will not work if I try to use nn.
Luckily, the net I am implementing this on is a super simple Feed-Forward MLP, so I will try implementing it by hand via Torch Tensors and multiplication, and post it here (for people having the same issue in the future, I will detail what parts break my code). Out of curiosity, do you think an elementwise like nn.tanh will also break the computational graph, or am I going to need to program in my own non-linearities as well?