I’m retraining resnet101 for an image classification task, and observe that my models behave differently in eval mode if it has previously been run in training mode. Here is a code example:
from torchvision import models
import torch
from PIL import Image
from torchvision import transforms

transform = transforms.Compose([  # [1]
    transforms.Resize(256),  # [2]
    transforms.CenterCrop(224),  # [3]
    transforms.ToTensor(),  # [4]
    transforms.Normalize(  # [5]
        mean=[0.485, 0.456, 0.406],  # [6]
        std=[0.229, 0.224, 0.225]  # [7]
    )])

img = Image.open("C:/image.jpg")
img_t = transform(img)
batch_t = torch.unsqueeze(img_t, 0).cuda()
resnet = models.resnet101(pretrained=True)
resnet.cuda()
resnet.eval()
print(max(resnet(batch_t)[0]))
# tensor(13.7692, device='cuda:0', grad_fn=<SelectBackward>)
# Peforming a model evaluation in training mode:
resnet.train()
dummy = resnet(batch_t)
# Switching back to eval mode, and the model output is different:
resnet.eval()
print(max(resnet(batch_t)[0]))
# tensor(13.2980, device='cuda:0', grad_fn=<SelectBackward>)

Apparently, running the model in training mode (without updating anything) changes the behavior of the model in eval mode.
I assume I’m missing something, but I wouldn’t expect the statements
resnet.train()
dummy = resnet(batch_t)
to make any changes to the model?
image750×1000 91.8 KB