Solved by hxuaj in post #2 
              
              
                The copy_ function should be:


m.weight.data.copy_(random_weight(m.weight.data.size())) 


The weight shape of nn.Linear in PyTorch is (out_features, in_features)! 
So in random_weighet, the fan_in is out_features. 
The params didn’t update is because the initialization weights were divided by the…