Recently I rebuild my caffe code with pytorch and got a much worse performance than original ones. Also I find the converge speed is slightly slower than before. When I check the initialization of model, I notice that in caffe’s BN(actually scale layer) layer parameter gamma is initialized with 1.0 while the default initialization in pytorch seems like random float numbers. In addition, when I initialize BN’s weight with 1.0 the training loss can drop faster. So I want to know what the default initialization is in pytorch’s BN layer. Anyone could please answer my question? Thanks a lot!
Regards
MeowLady