Bingchen_Liu:

If apply weight_init for G on cpu, the weight is indeed changed, but not as expected (0.02 vs 0.0099)


I cannot reproduce it and get a std of the expected ~0.02 for all weight parameters after running:
netG1.apply(weights_init)
for name, param in netG1.named_parameters():
    print(name, param.std())




 Bingchen_Liu:

If apply weight_init for G after G is on gpu, then this won’t change anything, although the model weights are indeed changed.


How are you checking, if the output changed? Note that a visual comparison might seem to look equal, but e.g. if your model is saturating after the initialization, the output might look equal but would be calculated in a different manner.