Hi, I am now using fixed seed.
seed = 3
random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False

Now, I am seeing exactly the same test accuracy every time but that is 1% less than the original.
The saved model produced 85.75% test accuracy. Now after reloading it is producing 84.75% every time I run the test set.