I have an interesting observation. If I save a model by using torch.save(model.state_dict(), “models.pth”) and reload it. Then the test accuracy changes. Here is the code I am using
#save the model
CUDA_DEVICE = 1
net = torch.load("wideresnet.pth")
torch.save(net.state_dict(), "wideresnetState.pth")

# reload it
net1 = wrn.WideResNet(depth=28, num_classes = 10, widen_factor=10, dropRate=0.0)
net1 = torch.nn.DataParallel(net1, device_ids=[CUDA_DEVICE]).cuda(CUDA_DEVICE)
net1.load_state_dict(torch.load("wideresnetState.pth"))

The test accuracy of “net” is 96.29% while the the test accuracy of “net1” is 94.99%.