The code looks generally alright.
There is one issue, which might create a growing memory usage and could thus trigger an out of memory error.
In these lines of code it seems you are directly storing the loss, which is still attached to the computation graph:
loss_train += loss 
l.append(loss_train)

Store the detached tensor, if the loss list is just used to calculate the average loss etc.:
loss_train += loss.item()

Have you tried to unfreeze the model and train it completely?
If so, what was the highest accuracy you could achieve?
Note that the pretrained models were trained on ImageNet12. The input images were thus normalized with:
normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                 std=[0.229, 0.224, 0.225])

While it seems you’ve calculated the image statistics manually based on your dataset or where do these values come from?