Okay, that is what I suspected. Unfortunately, I need to keep weight_current as an attribute of the class, so removing self. is not an option. Is there anything else I can do?
Is there a particular reason pytorch decided to implicitly register a new Parameter after assignment of a Parameter to an attribute in a nn.Module derived class? It seems to me that in most cases it would be preferable to do that explicitly.