Could this have something to do with the fact that in the second example you are using the pretrained model, which possibly changes the dimensions of the input the model expects? What happens if you try using pretrained=True in the first example as well (or remove pretrained=True in the second example)?