Since you model is initialized with random values, try to set the seed, so that the pseudorandom number generator generates the same “random” values for every run.
np.random.seed(SEED) # if numpy is used
torch.manual_seed(SEED)
if use_cuda:
    torch.cuda.manual_seed(SEED)

Also, some differences might occur when you are using CUDA, since some operations are not deterministic as fasr as I know.