In general, if you have a different number of output classes than the original network, the pretrained model is not going to work as you expect. You either have fewer classes, but they don’t mean the same thing as the original, or you have more classes, and the additional weights are simply going to be automatically generated (if you specified an initialization or use the default initialization).
This is why fine-tuning is often required. See this official tutorial 67 for detailed explanations.