Do optimizers just use the name of the parameters to track 'em? I’m looking to use register_forward_pre_hook for equalized learning rate as described in the progressive GAN 2 paper. In simple terms, the weight is divided by a constant after every learning step.
In this case, if I simply divide the weights in register_forward_pre_hook by say 2, how will it affect optimizer behavior?