You are fine-tuning all parameters using the pretrained weights as the initial parameters.
The difference is that you won’t initialize them randomly using a torch.nn.init method, but start from the already pretrained parameters.
If you are using a similar dataset to the one used for pretraining, the model parameters will be already quite good. Changing all parameters “a bit” will fine-tune the model so that it hopefully works better for your new dataset.