I see, I had not seen this detail about volatile.
This might warrant a new issue but since it’s on the same problem of training with gradients in the loss… On my second batch that I train I get:
RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.
Each batch loop now does roughly (cutting out details):
optimizer.zero_grad()
y = model.forward(x)
grads = torch.autograd.grad(y, x, torch.ones_like(y), create_graph=True) # to get the gradients 
loss = realgrad - grads
loss.backward() # to backprop the errors. The second time (second loop) this is called it errors
optimizer.step()

I don’t understand how I could get an error on the second batch while it works fine on the first batch.