The copy_ function should be:


m.weight.data.copy_(random_weight(m.weight.data.size()))


The weight shape of nn.Linear in PyTorch is (out_features, in_features)!
So in random_weighet, the fan_in is out_features.
The params didn’t update is because the initialization weights were divided by the number of out_features, which led gradient flow to vanish.