I tried using the following model for feature extraction instead, which would maintain the original layer connection from the original Resnet, and got slightly better training accuracy, but results were almost the same:
import torch.nn as nn
import torchvision.models as models

resnet = models.resnet152(pretrained=True)
num_ftrs_resnet = resnet.fc.in_features
for param in resnet.parameters():
    param.requires_grad = False
resnet.fc = nn.Flatten()