You understood me correctly. I’m not really interested in the grads, but what the optimizer computes when we call .step(). So, I guess you don’t see an efficient way of getting those values except than looking at the weights before, and after the update?