I suspect the backward pass might be broken somehow.
The only difference between my implementation of weight standardization hook function and Pytorch1.6.0’s weight normalization is within the compute_weight() function, where my implementation is explicitly done in python meanwhile Pytorch1.6.0’s implementation wraps the actual computation in another C level _weight_norm() function.