Solved by ptrblck in post #12 
              
              
                Thanks for the code. 
It seems your approach is similar to weight_norm, which uses the setattr and getattr to modify the weight parameter so you could try to adapt WeightNorm to your use case. 
Alternatively, it might be easier to use the functional API, which would define a self.weight parameter, m…