fang150:

Thanks Tomas. I am using loss.backward() to accumulate gradients, I also try to let retain_graph =True.


Yeah, but if you want to take derivatives of quantities involving the gradients, you need create_graph=True 88, too.
Best regards
Thomas