Hi,
I don’t think the inplace is the main problem here (even though it might still be later on).
The main issue is that optimizers are not differentiable. So it is expected that you won’t be able to differentiate through them.
Writing it manually will work if you do not use .data (which you should never use) that is similar to a .detach(). Meaning that you explicitly ask autograd not to track this op.