So I’m not exactly sure I understand what you are aiming at, but to me it looks uncommon that you would have two parameters “for the same thing”.
I don’t know nothing about quantization of weights and haven’t read a thing in the literature on it, but if you want to quantize the weight and keep the unquantized weight around for training, my initial approach would be to not have the quantized weight as a parameter (it doesn’t make much sense to have a gradient if you aim at discrete values), but rather having the unquantized (“raw”) weight as a parameter, i.e. something like
self.weight_raw = Parameter(something)

in the setup and then
quantized_weight = self.weight_raw + (torch.round(self.weight_raw)-self.weight_raw).detach()

(taking torch.round to quantize the function). And then do the calculculation with quantized_weight.
This way, you will do the forward with the quantized weight, but get the gradient of the unquantized weight for updates and also update that during training.
Admittedly, this could be totally of because I misunderstood what you want to do.
(If it is useful, here is the credit: I learnt that trick when @hughperkins shared it in the context of sampling (gumbel softmax). I used that for shake-shake networks 10.)
Best regards
Thomas