I think you would need clone because it’s likely that when I call g.backward(), the .grad attributes will be changed.
Actually I am not sure what do you mean by calculating and reassigning.
Regarding “The better way probably is writinh your own optimizer that extends torch.optim.Optimizer.”
There is no issue with extending the optimizer. The question is how to extend it. That’s exactly what I asked.