Solved by ptrblck in post #4 
              
              
                You are fine-tuning all parameters using the pretrained weights as the initial parameters. 
The difference is that you won’t initialize them randomly using a torch.nn.init method, but start from the already pretrained parameters. 
If you are using a similar dataset to the one used for pretraining, t…