I originally had resorted to using .data because of the error a leaf Variable that requires grad has been used in an in-place operation. But I see that with no_grad() takes of this problem!
As for the states of the optimizer, that is another challenge I’m working on by building optimizer specific to attacks. Which is only tangentially related to this topic.
Thanks!