I really can’t help you a lot without knowing what “other Variables” are you referring to? Are you optimizing them too? If not, just doing zero_grad() should be enough. There’s no way the content of .grad of a Variable, can affect what gets accumulated into another .grad. That’s not how derivatives work (of course I’m talking in a context where we don’t have multi backward).