You might want to see this thread Non Reproducible result with GPU.
Also, if you are using GPU and then you want set cudnn backend in benchmark mode. Also, set the random seeds.
Add the following lines at the start of your script
import torch
from torch.backends import cudnn
import numpy as np
np.random.seed(123)
torch.manual_seed(123)
if args.cuda:
torch.cuda.manual_seed(123)
cudnn.benchmark = True