I stack a network and have trained it well. Now, I test inference time of network that initialize with pre-trained weight, but it slower than initialization with random value in pytorch 0.4.0.
Then, I check the weight in the two condition.
Case A, initialize with random value as follow:
It seems that fixed-point;

image.jpg1300×314 150 KB

CaseB, initialize with pre-trained weight as follow:
It seems that float-point;
image.jpg1280×514 256 KB

I’m not very good at floating-point arithmetic in pytorch or other framework, but I intuition guess that there maybe some issue?
Thank you for your time and suggestion.